-- Logs begin at Mon 2022-10-24 08:03:16 UTC, end at Mon 2022-10-24 09:13:49 UTC. -- Oct 24 08:42:06.753594 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 24 08:42:06.764179 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 24 08:42:06.764999 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 24 08:42:06 2022] *** Oct 24 08:42:06.764999 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 24 08:42:06.764999 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 24 08:42:06.764999 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: nodename: dv-cinder-master-3-iscsi Oct 24 08:42:06.764999 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: machine: x86_64 Oct 24 08:42:06.764999 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: clock source: unix Oct 24 08:42:06.764999 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: pcre jit disabled Oct 24 08:42:06.764999 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: detected number of CPU cores: 4 Oct 24 08:42:06.764999 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: current working directory: / Oct 24 08:42:06.764999 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: detected binary path: /usr/bin/uwsgi-core Oct 24 08:42:06.764999 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: your processes number limit is 47680 Oct 24 08:42:06.764999 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: your memory page size is 4096 bytes Oct 24 08:42:06.765719 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: detected max file descriptor number: 2048 Oct 24 08:42:06.765719 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: lock engine: pthread robust mutexes Oct 24 08:42:06.765719 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: thunder lock: enabled Oct 24 08:42:06.765719 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 24 08:42:06.765719 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 24 08:42:06.806124 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: Python main interpreter initialized at 0x55ee9c2f89c0 Oct 24 08:42:06.806124 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: python threads support enabled Oct 24 08:42:06.806124 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: your server socket listen backlog is limited to 100 connections Oct 24 08:42:06.806124 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: your mercy for graceful operations on workers is 90 seconds Oct 24 08:42:06.806542 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: mapped 403077 bytes (393 KB) for 2 cores Oct 24 08:42:06.806542 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: *** Operational MODE: preforking *** Oct 24 08:42:06.806824 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: *** uWSGI is running in multiple interpreter mode *** Oct 24 08:42:06.806824 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: spawned uWSGI master process (pid: 115368) Oct 24 08:42:06.807041 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 24 08:42:06.808196 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: spawned uWSGI worker 1 (pid: 115369, cores: 1) Oct 24 08:42:06.808196 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: spawned uWSGI worker 2 (pid: 115370, cores: 1) Oct 24 08:42:06.808196 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 24 08:42:07.480976 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 24 08:42:07.488027 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 24 08:42:08.576787 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 08:42:08.576787 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: from cryptography.utils import int_from_bytes Oct 24 08:42:08.577430 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 08:42:08.577430 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: from cryptography.utils import int_from_bytes Oct 24 08:42:08.582558 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 08:42:08.582558 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: from cryptography.utils import int_from_bytes Oct 24 08:42:08.583150 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 08:42:08.583150 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: from cryptography.utils import int_from_bytes Oct 24 08:42:11.168126 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 24 08:42:11.168718 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 24 08:42:11.168967 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 24 08:42:11.169337 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 24 08:42:11.169597 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 24 08:42:11.169990 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.170265 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 24 08:42:11.170656 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.170741 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 24 08:42:11.170830 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.170918 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.171017 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 24 08:42:11.171253 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.171357 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 24 08:42:11.171569 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.171676 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 24 08:42:11.171891 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.171978 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.172236 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.172345 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.172568 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.172661 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.172828 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.172925 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.173145 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.173236 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.173474 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.173590 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.173796 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.173883 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.174105 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.174192 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.174416 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.174515 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.174668 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.174753 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.174976 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.175061 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.175295 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.175394 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.175610 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.175704 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.175908 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.176118 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.176547 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.176547 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.176728 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.176835 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.176923 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.177150 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.177245 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.177448 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.177532 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.177797 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.177950 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.178111 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.178310 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.178553 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.179460 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.179696 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.179900 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.180045 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.180155 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.180376 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.180496 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.180688 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.180819 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.180997 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.181134 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.181314 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.181447 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.181617 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.181870 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.181979 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.182181 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.182275 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.182495 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.182669 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.182819 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.182934 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.183144 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.183291 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.183457 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.183574 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.183939 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.184040 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.184193 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.184278 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.184492 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.184598 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.184810 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.184899 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.185124 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.185220 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.185425 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.185524 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.185736 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.185821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.186050 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.186139 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.186363 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.186463 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.186672 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.186759 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.186925 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.187012 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.187247 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.187442 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.187605 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.187755 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.187918 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.188079 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.188280 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.188431 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.188594 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.188783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.188934 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.189091 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.189259 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.189419 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.189577 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.189725 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.189872 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.190072 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.190221 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.190381 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.190538 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.190686 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.190869 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.191021 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.191152 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.191280 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.191410 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.191550 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.191702 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.191833 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.191967 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.192105 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.192285 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.192410 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.192547 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.192547 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.192784 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.192909 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.193033 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.193158 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.193293 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.193450 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.193596 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.193684 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.193792 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.39 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.193871 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.193966 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.39 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.194818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.194818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.195107 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.195253 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.195394 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.195528 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.195656 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.195791 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.195917 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.39 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.196042 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.196220 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.39 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.196350 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.196479 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.196560 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.196627 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.196826 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.197159 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.197318 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.197455 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.197539 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.197618 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.197711 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.197805 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.197936 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.198034 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.198233 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.198334 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.198542 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.198652 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.198862 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.198961 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.199170 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.199267 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.199480 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.199636 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.199789 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.199886 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.200090 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.200336 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.200446 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.200528 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.200723 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.200823 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.201019 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.201114 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.201324 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.201424 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.201625 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.201734 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.201920 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.202066 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.202217 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.202310 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.202518 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.202606 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.202819 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.202923 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.203121 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.203209 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.203419 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.203507 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.203721 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.203822 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.204021 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.204108 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.204346 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.204435 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.204648 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.204814 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.204946 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.205046 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.205260 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.205354 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.205565 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.205669 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.205865 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.205978 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.206159 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.206256 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.206459 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.206556 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.206758 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.206845 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.207070 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.207157 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.207371 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.207509 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.207672 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.207759 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.207988 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.208074 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.208312 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.208420 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.208612 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.208700 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.208923 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.209010 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.209226 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.209314 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.209555 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.209635 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.209861 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.210172 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.210374 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.210498 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.210678 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.210780 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.210991 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.211096 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.211291 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.211392 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.211596 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.211706 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.211901 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.212007 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.212225 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.212325 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.212530 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.212639 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.212829 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.212935 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.213135 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.213234 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.213433 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.213530 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.213732 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.213844 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.214026 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.214127 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.214342 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.214449 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.214659 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.214770 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.214963 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.215059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.215265 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.215376 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.215564 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.215688 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.215876 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:42:11.215975 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.216230 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.216329 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.216561 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.216667 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.216898 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.216996 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.217206 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.217304 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.217531 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.217636 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.217854 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.217953 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.218164 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.218261 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.218473 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.218571 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.218782 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.218888 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.219105 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.219226 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.219411 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.219497 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.219729 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.219837 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.220018 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.220116 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.220357 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.220455 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.220673 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.220772 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.221009 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.221114 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.221297 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.221437 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.221605 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.221697 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.221918 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.222019 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.222224 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.222358 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.222536 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.222686 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.222863 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.222970 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.223172 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.223322 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.223460 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.223618 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.223797 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.223950 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.224123 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.224278 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.224453 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.224592 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.224771 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.224918 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.225664 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.225801 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.225998 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.226210 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.226375 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.226608 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.226743 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.226921 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.227061 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.227233 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.227453 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.227604 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.227739 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.227878 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.228014 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.228176 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.228322 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.228473 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.228612 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.228797 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.228932 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.229109 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.229257 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.229417 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.229556 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.229726 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.229865 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.230034 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.230183 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.230345 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.230434 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.230654 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.230792 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.230961 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.231051 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.231267 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.231355 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.231580 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.231683 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.231886 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.231976 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.232219 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.232323 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.232532 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.232620 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.232848 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.232937 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.233156 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.233244 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.233459 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.233546 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.233746 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.233842 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.234046 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.234136 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.234351 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.234455 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.234650 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.234738 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.234962 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.235049 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.235276 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.235374 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.235576 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.235668 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.235899 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.235994 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.236229 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.236457 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.236561 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.236642 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.236835 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.236933 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.237140 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.237240 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.237447 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.237592 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.237733 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.237868 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.238044 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.238138 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.238344 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.238433 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.238658 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.238780 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.238966 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.239067 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.239268 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.239393 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.239586 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.239683 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.239884 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.239982 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.240211 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.240370 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.240530 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.240765 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.241460 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.241771 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.242016 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.242180 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.242351 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.242508 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.242674 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.242816 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.242971 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.243126 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.243266 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.243410 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.243559 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.243701 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.243851 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.243989 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.244177 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.244284 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.244489 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.244581 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.244795 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.244893 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.245098 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.245197 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.245403 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.245494 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.245712 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.245856 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.246035 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.246126 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.246352 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.246448 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.246652 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.246739 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.246967 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.247362 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.247502 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.247645 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.247804 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.247903 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.248107 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.248265 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.248434 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.248531 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.248737 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.248849 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.249033 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.249160 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.249336 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.249472 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.249650 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.249750 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.249956 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.250046 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.250256 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.250395 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.250557 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.250655 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.250855 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.250957 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.251195 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.251285 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.251499 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.251654 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.251827 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.251932 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.252175 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.252330 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.252538 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.252641 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.252804 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.252904 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.253115 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.253212 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.253428 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.253528 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.253734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.253840 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.254041 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.254137 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.254400 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.254504 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.254658 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.254798 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.254975 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.255072 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.255283 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.255374 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.255599 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.255688 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.255906 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.255995 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.256248 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.256371 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.256623 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.256937 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.257099 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.257253 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.257423 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.257577 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.257731 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.257893 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.258048 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.258216 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.258390 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.258529 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.258724 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.258858 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.259042 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.259171 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.259369 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.259478 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.259679 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.259782 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.259995 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.260094 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.260330 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.260433 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.260723 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.260808 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.260975 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.261083 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.261244 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.261356 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.261562 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.261680 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.261872 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.262027 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.262179 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.262320 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.262472 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.262719 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.262863 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.263010 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.263148 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.263287 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.263458 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.263627 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.263807 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.263973 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.264206 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.264371 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.264555 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.264699 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.264894 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.265068 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.265212 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.265384 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.265549 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.265698 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.265868 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.266007 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.266173 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.266338 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.266477 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.266649 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.266809 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.266955 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.267091 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.267228 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.267365 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.267516 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.267660 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.267809 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.267949 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.268085 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.268270 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.268422 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.268560 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.268696 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.268834 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.268972 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.269106 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.269252 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.269388 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.269582 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.269715 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.269853 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.270002 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.270139 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.270276 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.270411 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.270567 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.270711 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.270859 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.270998 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.271169 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.271348 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.271506 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.271642 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.271790 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.271924 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.272161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.272300 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.272435 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.272581 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.272581 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.272901 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.273036 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.273214 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.273348 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.273530 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.273671 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.273845 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.273982 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.274174 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.274310 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.274473 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.274571 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.274780 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.274953 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.275090 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.275182 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.275452 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.275644 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.275818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.275973 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.276062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.276214 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.276322 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.276450 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.276653 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.276743 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.276994 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.277208 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.277306 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.277544 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.277716 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.277870 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.278048 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.278189 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.278367 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.278482 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.278558 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.278834 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.278950 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.279060 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.279159 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.279368 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.279476 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.279678 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.279793 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.279985 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.280092 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.280330 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.280420 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.280640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.280766 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.280956 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.281138 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.281310 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.281523 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.281705 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.281873 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.282032 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.282180 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.282308 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.282446 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.282545 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.282754 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.282872 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.283112 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.283223 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.283350 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.283447 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.283657 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.283777 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.283973 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.284082 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.284319 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.284470 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.284608 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.284763 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.284916 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.285025 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.285192 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.285306 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.285501 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.285604 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.285804 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.285917 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.286105 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.286253 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.286402 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.286491 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.286710 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.286800 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.287009 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.287096 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.287309 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.287416 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.288365 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.288507 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.288606 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.288922 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.289089 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.289230 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.289407 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.289561 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.289722 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.289948 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.290101 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.290262 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.290409 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.290565 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.290710 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.290853 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.290997 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.291150 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.291288 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.291435 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.291578 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.291720 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.291878 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.292006 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.292177 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.292308 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.292498 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.292629 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.292779 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.292909 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.293076 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.293213 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.293420 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.293558 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.293724 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.293860 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.294034 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.294182 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.294357 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.294506 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.294645 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.294777 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.294953 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.295088 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.295255 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.295431 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.295563 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.295704 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.295862 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.295965 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.296185 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.296293 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.296504 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.296600 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.296812 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.296900 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.297121 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.297220 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.297430 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.297517 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.297730 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.297817 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.298037 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.298141 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.298339 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.298436 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.298648 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.298735 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.298955 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.299044 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.299256 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.299344 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.299621 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.299729 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.299863 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.299960 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.300235 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.300472 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.300567 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.300658 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.300777 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.300867 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.301079 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.301166 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.301393 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.301482 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.301695 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.301785 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.302130 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.302358 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.302497 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.302629 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.302629 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.302927 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.303176 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.303916 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.304063 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.304063 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.304350 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.304459 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.304584 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.304787 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.304947 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.305105 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.305238 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.305420 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.305542 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.305758 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.305881 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.306174 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.306308 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.306617 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.306813 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.307023 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.307237 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.307370 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.307553 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.307759 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.307948 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.308168 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.308371 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.308371 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.308724 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.308910 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.309095 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.309277 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.309477 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.309678 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.309866 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.310052 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.310234 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.310418 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.310614 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.310798 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.310984 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.311168 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.311348 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.311547 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.311734 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.311921 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.312209 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.312522 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.312710 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.312958 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.313187 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.313377 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.313556 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.313741 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.313923 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.314119 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.314309 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.314494 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.314691 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.314876 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.315060 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.315255 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.315451 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.315639 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.315836 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.315976 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.316123 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.316338 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.316477 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.316610 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.316744 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.316879 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.317038 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.317173 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.317309 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.317444 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.317577 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.317725 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.317875 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.318010 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.318142 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.318275 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.318407 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.318540 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.318688 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.318869 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.319003 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.319136 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.319279 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.319457 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.319654 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.319814 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.319975 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.320104 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.320315 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.320508 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.320648 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.320822 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.321004 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.321172 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.321172 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.321454 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.321454 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.321454 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.321856 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.321987 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.322137 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.322290 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.322558 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.322737 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.322904 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.323043 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.323277 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.323419 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.323584 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.323724 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.323859 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.323993 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.324163 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.324304 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.324453 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.324583 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.324710 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.324848 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.324988 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.325126 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.325253 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.325253 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.325490 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.325619 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.325771 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.325906 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.326050 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.326178 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.326315 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.326507 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.326645 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.326775 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.326902 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.327031 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.327170 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.327309 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.327438 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.327565 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.327696 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.327835 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.327963 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.327963 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.328271 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.328271 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.328511 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.328637 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.328762 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.328911 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.329038 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.329182 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.329310 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.329438 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.329575 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.329715 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.329842 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.329976 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.330103 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.330228 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.330365 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.330488 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.330599 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.330738 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.330919 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.331053 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.331196 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.331320 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.331443 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.331570 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.331695 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.331819 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.331959 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.332086 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.332242 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.332374 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.332553 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.332684 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.332878 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.333041 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.333184 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.333361 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.333487 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.333661 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.333790 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.333918 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.334054 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.334183 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.334307 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.334475 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.334603 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.334742 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.334868 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.334994 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.335123 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.335253 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.335389 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.335389 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.335675 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.335806 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.335934 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.336069 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.336275 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.336486 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.336671 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.336805 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.336940 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.337074 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.337217 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.337350 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.337483 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.337616 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.337755 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.337886 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.338027 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.338162 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.338325 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.338463 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.338642 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.338834 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.338981 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.339113 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.339247 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.339376 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.339536 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.339673 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.339863 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.340000 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.340190 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.340345 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.340519 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.340676 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.340876 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.341022 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.341156 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.341290 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.341458 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.341617 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.341756 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.341897 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.342076 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.342223 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.342367 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.342525 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.342719 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.342869 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.343079 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.343230 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.343373 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.343588 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.343736 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.343876 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.344014 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.344257 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.344257 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.344534 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.344668 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.344806 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.344935 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.345062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.345215 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.345344 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.345474 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.345609 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.345743 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.345875 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.346031 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.346171 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.346307 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.346447 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.346595 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.346810 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.346971 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.347110 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.347240 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.347377 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.347506 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.347651 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.347789 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.347919 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.348047 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.348199 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.348357 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.348511 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.348652 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.348811 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.348986 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.349130 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.349290 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.349442 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.349609 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.349753 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.349936 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.350619 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.350850 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.351070 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.351268 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.351600 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.351787 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.351941 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.352125 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.352323 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.352461 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.352596 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.352779 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.352931 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.353130 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.353282 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.353421 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.353613 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.353754 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.353905 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.354055 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.354193 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.354329 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.354463 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.354598 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.354734 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.354883 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.355019 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.355151 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.355298 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.355432 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.355604 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.355741 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.356072 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.356274 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.356414 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.356576 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.356725 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.356934 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.357084 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.357221 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.357354 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.357503 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.357639 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.357774 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.357912 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.358051 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.358349 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.358603 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.358741 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.358869 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.358999 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.359140 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.359291 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.359427 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.359565 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.359698 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.359866 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.360005 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.360232 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.360365 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.360498 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.360632 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.360822 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.360995 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.361161 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.361295 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.361423 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.361553 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.361728 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.361863 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.362040 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.362187 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.362350 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.362488 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.362659 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.362812 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.363009 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.363149 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.363345 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.363483 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.363617 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.363751 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.363887 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.364113 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.364305 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.364446 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.364594 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.364741 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.364887 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.365058 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.365198 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.365371 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.365661 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.366327 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.366493 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.366640 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.367053 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.367198 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.367552 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.367699 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.367837 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.367975 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.368123 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.368386 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.368524 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.368672 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.368807 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.368940 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.369087 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.369235 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.369376 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.369538 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.369731 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.369873 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.370005 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.370154 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.370288 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.370464 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.370594 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.370789 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.370943 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.371150 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.371363 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.371513 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.371661 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.371834 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.371984 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.372429 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.372568 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.372754 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.372894 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.373081 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.373235 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.373417 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.373566 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.373698 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.373899 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.374048 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.374180 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.374313 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.374443 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.374627 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.374777 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.374930 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.375072 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.375248 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.375389 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.375567 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.375709 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.375893 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.376060 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.376227 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.376369 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.376530 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.376684 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.376839 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.376981 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.377207 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.377357 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.377513 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.377681 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.377828 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.378011 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.378343 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.378497 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.378672 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.378834 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.378995 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.379146 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.379310 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.379444 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.379642 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.379809 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.379944 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.380081 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.380267 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.380418 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.380565 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.380700 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.380879 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.381032 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.381171 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.381876 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.381876 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.382165 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.382315 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.382465 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.382608 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.382750 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.382750 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.383032 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.383171 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.383295 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.383433 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.383634 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.383789 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.383978 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.384121 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.384322 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.384472 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.384623 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.384760 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.384895 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.385028 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.385028 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.385269 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.385409 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.385539 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.385666 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.385810 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.385948 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.386100 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.386259 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.386405 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.386543 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.386682 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.386829 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.386969 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.387125 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.387324 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.387481 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.387619 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.387756 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.387913 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.388063 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.388225 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.388363 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.388517 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.388657 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.388810 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.388946 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.389081 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.389216 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.389355 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.389490 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.389639 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.389775 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.389931 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.390071 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.390209 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.390342 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.390492 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.390627 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.390761 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.390898 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.391036 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.391182 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.391323 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.391460 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.391724 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.391872 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.392045 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.392269 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.392421 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.392570 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.392708 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.392846 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.392980 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.393126 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.393263 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.393377 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.393563 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.393710 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.393878 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.394032 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.394236 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.394376 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.394543 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.394683 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.394833 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.394971 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.395164 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.395306 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.395472 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.395616 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.395865 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.396020 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.396214 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.396354 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.396492 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.396632 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.396789 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.396964 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.397099 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.397335 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.397521 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.397687 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.397959 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.398154 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.398290 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.398574 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.398723 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.398873 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.399013 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.399189 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.399331 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.399510 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.399665 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.399851 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.400008 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.400281 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.400424 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.400584 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.400720 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.400860 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.400992 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.401123 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.401266 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.401398 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.401546 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.401679 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.401811 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.401987 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.402142 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.402299 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.402441 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.402612 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.402759 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.402922 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.403048 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.403235 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.403376 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.403562 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.403687 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.403867 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.404165 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.404318 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.404472 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.404600 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.404783 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.404914 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.405093 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.405233 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.405404 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.405531 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.405720 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.405848 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.406062 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.406214 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.406385 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.406486 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.406703 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.406793 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.407028 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.407119 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.407348 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.407436 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.407693 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.407804 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.407967 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.408058 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.408304 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.408397 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.408619 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.408707 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.408937 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.409028 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.409267 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.409365 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.409581 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.409679 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.409885 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.409972 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.410201 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.410512 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.410600 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.410822 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.410921 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.411138 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.411238 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.411454 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.411576 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.411758 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.411850 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.412067 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.412246 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.412404 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.413138 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.413325 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.413416 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.413662 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.413873 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.414024 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.414208 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.414351 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.414496 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.414639 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.414785 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.414930 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.415089 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.415235 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.415373 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.415507 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.415645 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.415776 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.415931 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.416076 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.416250 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.416387 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.416528 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.416686 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.416869 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.416994 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.417114 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.417371 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.417530 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.417687 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.417826 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.417923 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.417994 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.418211 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.418302 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.418519 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.418618 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.418825 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.418915 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.419131 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.419271 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.419438 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.419591 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.419729 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.419885 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.420022 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.420253 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.420348 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.420426 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.420636 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.420756 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.420939 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.421073 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.421242 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.421390 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.421552 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.421707 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.421855 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.422024 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.422161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.422338 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.422464 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.422660 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.422789 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.422991 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.423087 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.423323 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.423422 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.423635 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.423821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.423964 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.424057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.424331 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.424479 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.424632 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.424736 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.424951 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.425040 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.425260 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.425357 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.425582 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.425756 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.425896 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.425984 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.426209 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.426314 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.426523 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.426624 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.426847 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.426983 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.427148 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.427239 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.427459 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.427548 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.427773 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.427898 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.428084 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.428283 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.428987 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.429089 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.429312 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.429410 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.429646 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.429734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.429973 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.430061 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.430282 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.430380 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.430610 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.430708 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.430931 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.431025 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.431253 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.431374 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.431572 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.431751 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.431885 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.432010 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.432241 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.432337 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.432558 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.432664 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.432869 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.432959 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.433184 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.433282 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.433431 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=115370) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 24 08:42:11.433547 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.433798 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.434108 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.434419 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.434732 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.435048 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.435367 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.435703 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.436016 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.436352 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.436685 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.437016 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.437362 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.437677 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.438025 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.438352 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.438665 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.438977 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.439281 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:42:11.439527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=115369) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 24 08:42:11.588183 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO dbcounter [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] Registered counter for database nova_api Oct 24 08:42:11.594463 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO dbcounter [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] Registered counter for database nova_api Oct 24 08:42:11.610430 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG oslo_db.sqlalchemy.engines [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=115370) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:42:11.614908 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_db.sqlalchemy.engines [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=115369) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:42:11.617341 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG dbcounter [-] [115370] Writer thread running {{(pid=115370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:42:11.621937 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG dbcounter [-] [115369] Writer thread running {{(pid=115369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:42:11.675408 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.context [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),0c69bc68-c2d9-408a-9288-c4e45fffaab5(cell1) {{(pid=115370) load_cells /opt/stack/nova/nova/context.py:464}} Oct 24 08:42:11.678776 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.context [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),0c69bc68-c2d9-408a-9288-c4e45fffaab5(cell1) {{(pid=115369) load_cells /opt/stack/nova/nova/context.py:464}} Oct 24 08:42:11.679361 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG oslo_concurrency.lockutils [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=115370) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:42:11.679828 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG oslo_concurrency.lockutils [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=115370) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:42:11.681482 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG oslo_concurrency.lockutils [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=115370) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:42:11.682199 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG oslo_concurrency.lockutils [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=115370) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:42:11.682589 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:42:11.682685 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG oslo_concurrency.lockutils [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115370) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:42:11.683054 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:42:11.684663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:42:11.684767 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG oslo_concurrency.lockutils [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=115370) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:42:11.685372 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:42:11.685806 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:42:11.686658 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO dbcounter [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] Registered counter for database nova_cell0 Oct 24 08:42:11.687921 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:42:11.689834 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO dbcounter [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] Registered counter for database nova_cell0 Oct 24 08:42:11.690854 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO dbcounter [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] Registered counter for database nova_cell1 Oct 24 08:42:11.693903 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO dbcounter [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] Registered counter for database nova_cell1 Oct 24 08:42:11.699044 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG oslo_db.sqlalchemy.engines [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=115370) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:42:11.700527 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG oslo_db.sqlalchemy.engines [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=115370) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:42:11.702398 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_db.sqlalchemy.engines [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=115369) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:42:11.703698 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_db.sqlalchemy.engines [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=115369) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:42:11.708414 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG dbcounter [-] [115370] Writer thread running {{(pid=115370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:42:11.708757 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG dbcounter [-] [115370] Writer thread running {{(pid=115370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:42:11.711499 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG dbcounter [-] [115369] Writer thread running {{(pid=115369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:42:11.716473 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG dbcounter [-] [115369] Writer thread running {{(pid=115369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:42:11.857197 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO dbcounter [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] Registered counter for database nova_cell0 Oct 24 08:42:11.861922 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO dbcounter [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] Registered counter for database nova_cell0 Oct 24 08:42:11.866803 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG oslo_db.sqlalchemy.engines [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=115370) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:42:11.871596 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_db.sqlalchemy.engines [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=115369) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:42:11.871991 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG dbcounter [-] [115370] Writer thread running {{(pid=115370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:42:11.876709 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG dbcounter [-] [115369] Writer thread running {{(pid=115369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:42:12.219428 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: WARNING keystonemiddleware.auth_token [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 24 08:42:12.220100 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: WARNING keystonemiddleware.auth_token [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 24 08:42:12.227340 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: WARNING keystonemiddleware.auth_token [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 24 08:42:12.227600 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: WARNING keystonemiddleware.auth_token [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 24 08:42:12.573208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: WARNING keystonemiddleware.auth_token [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 24 08:42:12.577798 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: WARNING keystonemiddleware.auth_token [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 24 08:42:12.579161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: WARNING keystonemiddleware.auth_token [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 24 08:42:12.583779 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: WARNING keystonemiddleware.auth_token [None req-e8a757ce-43bf-4025-88cc-9b332b1f100d None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 24 08:42:12.584550 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ee9c2f89c0 pid: 115369 (default app) Oct 24 08:42:12.587283 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:42:12.588510 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] 10.222.0.39 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.002203 Oct 24 08:42:12.589222 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 1/1] 10.222.0.39 () {50 vars in 788 bytes} [Mon Oct 24 08:42:12 2022] GET /compute/v2.1/ => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:42:12.589796 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ee9c2f89c0 pid: 115370 (default app) Oct 24 08:43:37.051242 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:43:37.052196 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001565 Oct 24 08:43:37.054844 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 2/2] 10.222.0.39 () {54 vars in 931 bytes} [Mon Oct 24 08:43:37 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:43:37.070144 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG dbcounter [-] [115370] Writing DB stats nova_api:SELECT=3 {{(pid=115370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:37.071388 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG dbcounter [-] [115370] Writing DB stats nova_cell1:SELECT=3 {{(pid=115370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:37.072110 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG dbcounter [-] [115370] Writing DB stats nova_cell0:SELECT=3 {{(pid=115370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:37.072963 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG dbcounter [-] [115370] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=115370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:38.052752 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-57443d11-9881-42a3-bbed-fac310a195d3 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:43:38.088068 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-57443d11-9881-42a3-bbed-fac310a195d3 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 1.028084 Oct 24 08:43:38.089839 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 1/3] 10.222.0.39 () {60 vars in 1292 bytes} [Mon Oct 24 08:43:37 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 1030 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:43:40.794349 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:43:40.794871 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-63c640f6-f08d-4405-9e4b-a5715c4005fb None None] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001542 Oct 24 08:43:40.795719 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 3/4] 10.222.0.39 () {54 vars in 931 bytes} [Mon Oct 24 08:43:40 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:43:41.266192 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-a8f79a72-170e-41c9-817a-70dafc4b45fb admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "vcpus": 1, "name": "cirros256", "ram": 256, "id": "c1", "disk": 1, "rxtx_factor": 1.0, "swap": 0}} {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:43:41.313933 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-a8f79a72-170e-41c9-817a-70dafc4b45fb admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.511016 Oct 24 08:43:41.314512 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 2/5] 10.222.0.39 () {64 vars in 1283 bytes} [Mon Oct 24 08:43:40 2022] POST /compute/v2.1/flavors => generated 407 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:43:41.329263 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG dbcounter [-] [115369] Writing DB stats nova_api:SELECT=3 {{(pid=115369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:41.331347 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG dbcounter [-] [115369] Writing DB stats nova_cell0:SELECT=3 {{(pid=115369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:41.332163 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG dbcounter [-] [115369] Writing DB stats nova_cell1:SELECT=3 {{(pid=115369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:41.333035 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG dbcounter [-] [115369] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=115369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:41.540856 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-5875803f-2454-4c0d-89d5-ee12bde39b73 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:43:41.611919 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-5875803f-2454-4c0d-89d5-ee12bde39b73 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.287888 Oct 24 08:43:41.612535 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 4/6] 10.222.0.39 () {64 vars in 1336 bytes} [Mon Oct 24 08:43:41 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:43:44.305326 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-a8f79a72-170e-41c9-817a-70dafc4b45fb admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:43:44.306355 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-a8f79a72-170e-41c9-817a-70dafc4b45fb admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001489 Oct 24 08:43:44.306996 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 3/7] 10.222.0.39 () {54 vars in 931 bytes} [Mon Oct 24 08:43:44 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:43:45.001773 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-3517ea72-532b-4edb-ae5a-57ab71d1c836 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "name": "ds512M", "ram": 512, "id": "d1", "disk": 5, "swap": 0, "rxtx_factor": 1.0, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:43:45.048251 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-3517ea72-532b-4edb-ae5a-57ab71d1c836 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.734733 Oct 24 08:43:45.056220 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 5/8] 10.222.0.39 () {64 vars in 1283 bytes} [Mon Oct 24 08:43:44 2022] POST /compute/v2.1/flavors => generated 404 bytes in 736 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:43:45.063786 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-24e3d888-8f3b-4695-924b-0a73d22229bb admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:43:45.108714 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-24e3d888-8f3b-4695-924b-0a73d22229bb admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.049113 Oct 24 08:43:45.116676 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 4/9] 10.222.0.39 () {64 vars in 1336 bytes} [Mon Oct 24 08:43:45 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:43:47.783979 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-3517ea72-532b-4edb-ae5a-57ab71d1c836 admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:43:47.786614 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-3517ea72-532b-4edb-ae5a-57ab71d1c836 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003024 Oct 24 08:43:47.787034 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 6/10] 10.222.0.39 () {54 vars in 931 bytes} [Mon Oct 24 08:43:47 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:43:48.234193 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-fd676a98-c707-4bf7-8aa1-a5fc00961690 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 1024, "name": "ds1G", "id": "d2", "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "swap": 0, "disk": 10, "rxtx_factor": 1.0}} {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:43:48.257288 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-fd676a98-c707-4bf7-8aa1-a5fc00961690 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.463225 Oct 24 08:43:48.257840 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 5/11] 10.222.0.39 () {64 vars in 1283 bytes} [Mon Oct 24 08:43:47 2022] POST /compute/v2.1/flavors => generated 404 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 24 08:43:48.270813 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-e4d50269-dcc8-4d22-b0fd-b38ccca2d63d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:43:48.299926 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-e4d50269-dcc8-4d22-b0fd-b38ccca2d63d admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033258 Oct 24 08:43:48.301301 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 7/12] 10.222.0.39 () {64 vars in 1336 bytes} [Mon Oct 24 08:43:48 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:43:50.996724 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-fd676a98-c707-4bf7-8aa1-a5fc00961690 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:43:50.998648 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-fd676a98-c707-4bf7-8aa1-a5fc00961690 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002328 Oct 24 08:43:50.999257 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 6/13] 10.222.0.39 () {54 vars in 931 bytes} [Mon Oct 24 08:43:50 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:43:51.426735 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-901bf682-f3fc-4bad-a195-30b8d2ee0da3 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 2, "id": "d3", "disk": 10, "swap": 0, "rxtx_factor": 1.0, "ram": 2048, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds2G"}} {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:43:51.448189 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-901bf682-f3fc-4bad-a195-30b8d2ee0da3 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.441534 Oct 24 08:43:51.448852 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 8/14] 10.222.0.39 () {64 vars in 1283 bytes} [Mon Oct 24 08:43:51 2022] POST /compute/v2.1/flavors => generated 404 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 24 08:43:51.460695 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-26d50189-8c7e-40cd-80ea-7f60299ce7f8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:43:51.490586 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-26d50189-8c7e-40cd-80ea-7f60299ce7f8 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033352 Oct 24 08:43:51.491316 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 7/15] 10.222.0.39 () {64 vars in 1336 bytes} [Mon Oct 24 08:43:51 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:43:54.185562 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-901bf682-f3fc-4bad-a195-30b8d2ee0da3 admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:43:54.186842 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-901bf682-f3fc-4bad-a195-30b8d2ee0da3 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001842 Oct 24 08:43:54.187329 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 9/16] 10.222.0.39 () {54 vars in 931 bytes} [Mon Oct 24 08:43:54 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:43:54.638206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-cdecb6d8-3168-4015-b359-a944b6e6cfdb admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 4096, "disk": 20, "rxtx_factor": 1.0, "name": "ds4G", "os-flavor-access:is_public": true, "id": "d4", "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 4, "swap": 0}} {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:43:54.661803 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-cdecb6d8-3168-4015-b359-a944b6e6cfdb admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.466336 Oct 24 08:43:54.662370 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 8/17] 10.222.0.39 () {64 vars in 1283 bytes} [Mon Oct 24 08:43:54 2022] POST /compute/v2.1/flavors => generated 404 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 24 08:43:54.679334 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-f40ed10d-8d04-48a9-b424-ab252e2274b0 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:43:54.705222 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-f40ed10d-8d04-48a9-b424-ab252e2274b0 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032188 Oct 24 08:43:54.706000 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 10/18] 10.222.0.39 () {64 vars in 1336 bytes} [Mon Oct 24 08:43:54 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:43:57.399560 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-cdecb6d8-3168-4015-b359-a944b6e6cfdb admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:43:57.400624 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-cdecb6d8-3168-4015-b359-a944b6e6cfdb admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001506 Oct 24 08:43:57.401223 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 9/19] 10.222.0.39 () {54 vars in 931 bytes} [Mon Oct 24 08:43:57 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:43:57.859688 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-b1038d34-6fc4-4376-90f1-8cf1ed4301f6 admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:43:57.885513 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-b1038d34-6fc4-4376-90f1-8cf1ed4301f6 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.478016 Oct 24 08:43:57.886069 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 11/20] 10.222.0.39 () {60 vars in 1292 bytes} [Mon Oct 24 08:43:57 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:44:00.646632 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-cdecb6d8-3168-4015-b359-a944b6e6cfdb admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:00.647654 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-cdecb6d8-3168-4015-b359-a944b6e6cfdb admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001464 Oct 24 08:44:00.648066 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 10/21] 10.222.0.39 () {54 vars in 931 bytes} [Mon Oct 24 08:44:00 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:44:01.062052 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-85634f8a-baca-4a15-a4cd-c828a3c84c6d admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.tiny", "id": "1", "os-flavor-access:is_public": true, "vcpus": 1, "ram": 512, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 1, "rxtx_factor": 1.0}} {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:44:01.083749 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-85634f8a-baca-4a15-a4cd-c828a3c84c6d admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.429544 Oct 24 08:44:01.084325 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 12/22] 10.222.0.39 () {64 vars in 1283 bytes} [Mon Oct 24 08:44:00 2022] POST /compute/v2.1/flavors => generated 402 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:44:01.096694 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-beb70970-b1f8-4102-88d3-2b5ae599292a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:44:01.125769 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-beb70970-b1f8-4102-88d3-2b5ae599292a admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032874 Oct 24 08:44:01.126383 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 11/23] 10.222.0.39 () {64 vars in 1333 bytes} [Mon Oct 24 08:44:01 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:44:03.827485 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-85634f8a-baca-4a15-a4cd-c828a3c84c6d admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:03.828626 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-85634f8a-baca-4a15-a4cd-c828a3c84c6d admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001534 Oct 24 08:44:03.829023 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 13/24] 10.222.0.39 () {54 vars in 931 bytes} [Mon Oct 24 08:44:03 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:44:04.259637 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-7a58de1a-f343-4bfa-8fe8-7924ebbaec9a admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "id": "2", "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 2048, "vcpus": 1, "swap": 0, "name": "m1.small", "disk": 20}} {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:44:04.280515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-7a58de1a-f343-4bfa-8fe8-7924ebbaec9a admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.444329 Oct 24 08:44:04.281070 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 12/25] 10.222.0.39 () {64 vars in 1283 bytes} [Mon Oct 24 08:44:03 2022] POST /compute/v2.1/flavors => generated 405 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 24 08:44:04.297095 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-0082a831-6ed7-44b7-958c-e121021fccb3 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:44:04.326678 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-0082a831-6ed7-44b7-958c-e121021fccb3 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.035619 Oct 24 08:44:04.327259 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 14/26] 10.222.0.39 () {64 vars in 1333 bytes} [Mon Oct 24 08:44:04 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:44:07.150903 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-7a58de1a-f343-4bfa-8fe8-7924ebbaec9a admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:07.152553 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-7a58de1a-f343-4bfa-8fe8-7924ebbaec9a admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002047 Oct 24 08:44:07.152944 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 13/27] 10.222.0.39 () {54 vars in 931 bytes} [Mon Oct 24 08:44:07 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:44:07.610156 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-ba664b2f-bf16-4e74-8125-d28f73ff1360 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "3", "vcpus": 2, "disk": 40, "rxtx_factor": 1.0, "ram": 4096, "swap": 0, "name": "m1.medium", "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true}} {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:44:07.635109 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-ba664b2f-bf16-4e74-8125-d28f73ff1360 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.476038 Oct 24 08:44:07.635650 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 15/28] 10.222.0.39 () {64 vars in 1283 bytes} [Mon Oct 24 08:44:07 2022] POST /compute/v2.1/flavors => generated 406 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 24 08:44:07.648575 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-9e48af5c-4b54-4d22-afe2-d01956c6f487 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:44:07.653284 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG dbcounter [-] [115370] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=7,nova_api:SAVEPOINT=3,nova_api:RELEASE=3 {{(pid=115370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:44:07.680896 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-9e48af5c-4b54-4d22-afe2-d01956c6f487 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.036335 Oct 24 08:44:07.681792 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 14/29] 10.222.0.39 () {64 vars in 1333 bytes} [Mon Oct 24 08:44:07 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:44:10.372961 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-ba664b2f-bf16-4e74-8125-d28f73ff1360 admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:10.374000 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-ba664b2f-bf16-4e74-8125-d28f73ff1360 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001881 Oct 24 08:44:10.374567 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 16/30] 10.222.0.39 () {54 vars in 931 bytes} [Mon Oct 24 08:44:10 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:44:10.863121 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-9648d413-ddcd-4a0e-b343-963d580c32b1 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 8192, "id": "4", "rxtx_factor": 1.0, "name": "m1.large", "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 4, "os-flavor-access:is_public": true, "swap": 0, "disk": 80}} {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:44:10.882722 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-9648d413-ddcd-4a0e-b343-963d580c32b1 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.498825 Oct 24 08:44:10.888219 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 15/31] 10.222.0.39 () {64 vars in 1283 bytes} [Mon Oct 24 08:44:10 2022] POST /compute/v2.1/flavors => generated 405 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 24 08:44:10.901538 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-00398eec-6fa2-4d4d-a9a9-97ff896ca49a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:44:10.942769 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-00398eec-6fa2-4d4d-a9a9-97ff896ca49a admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.046796 Oct 24 08:44:10.944148 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 17/32] 10.222.0.39 () {64 vars in 1333 bytes} [Mon Oct 24 08:44:10 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:44:13.721335 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-9648d413-ddcd-4a0e-b343-963d580c32b1 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:13.724509 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-9648d413-ddcd-4a0e-b343-963d580c32b1 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.005656 Oct 24 08:44:13.725002 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 16/33] 10.222.0.39 () {54 vars in 931 bytes} [Mon Oct 24 08:44:13 2022] GET /compute/v2.1 => generated 388 bytes in 7 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:44:14.197789 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-084667ef-6f8c-4f88-835f-5b96e64ba453 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "disk": 160, "vcpus": 8, "id": "5", "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "ram": 16384, "name": "m1.xlarge", "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:44:14.204568 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG dbcounter [-] [115369] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=115369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:44:14.228120 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-084667ef-6f8c-4f88-835f-5b96e64ba453 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.496316 Oct 24 08:44:14.232219 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 18/34] 10.222.0.39 () {64 vars in 1283 bytes} [Mon Oct 24 08:44:13 2022] POST /compute/v2.1/flavors => generated 408 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:44:14.242227 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-b7cc3aa4-f3c1-48cd-b787-55793f0b17d9 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:44:14.270159 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-b7cc3aa4-f3c1-48cd-b787-55793f0b17d9 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031328 Oct 24 08:44:14.271460 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 17/35] 10.222.0.39 () {64 vars in 1333 bytes} [Mon Oct 24 08:44:14 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:44:30.921259 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-084667ef-6f8c-4f88-835f-5b96e64ba453 admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:30.922276 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-084667ef-6f8c-4f88-835f-5b96e64ba453 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001476 Oct 24 08:44:30.922679 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 19/36] 10.222.0.39 () {54 vars in 931 bytes} [Mon Oct 24 08:44:30 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:44:30.934405 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG dbcounter [-] [115370] Writing DB stats nova_api:SELECT=14,nova_api:SAVEPOINT=2,nova_api:INSERT=3,nova_api:RELEASE=2 {{(pid=115370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:44:31.341272 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-390ba72f-b9c3-49b0-b9ea-8d5de05bddb7 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:31.359805 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-390ba72f-b9c3-49b0-b9ea-8d5de05bddb7 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.429676 Oct 24 08:44:31.360286 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 18/37] 10.222.0.39 () {60 vars in 1292 bytes} [Mon Oct 24 08:44:30 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:44:32.698335 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-084667ef-6f8c-4f88-835f-5b96e64ba453 admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:32.700088 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-084667ef-6f8c-4f88-835f-5b96e64ba453 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002522 Oct 24 08:44:32.700871 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 20/38] 10.222.0.39 () {56 vars in 1074 bytes} [Mon Oct 24 08:44:32 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:44:33.257456 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-0bc89a0d-67f3-4c7e-91f4-4ffc7adeb7fd admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:33.262804 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.wsgi [None req-0bc89a0d-67f3-4c7e-91f4-4ffc7adeb7fd admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-0bc89a0d-67f3-4c7e-91f4-4ffc7adeb7fd admin admin] Request method failure captured: Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Accept: application/json Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Accept-Encoding: gzip, deflate Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Connection: keep-alive Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Content-Length: 0 Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Host: 10.222.0.39 Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Openstack-Api-Version: compute 2.92 Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Openstack-System-Scope: None Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: User-Agent: python-novaclient Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Auth-Token: gAAAAABjVlBwYOH1-QFpRRXmsHsQ4meUwJwvOo9I91tpjIcmTi3adsee4OnbL6YpHMlxkaRK7luHGNlQVnnwVbnGO201BQX6Ikm3KWFMoD676WkRE4r2YWSVCjuXC_skgIhMUrG4tJSDAZ4nX-nLqh0SC1f4f9aWnXrryU61MA6EdWrewliDuBQ Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Domain-Id: None Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Domain-Name: None Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Identity-Status: Confirmed Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Is-Admin-Project: True Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Openstack-Nova-Api-Version: 2.92 Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Project-Domain-Id: default Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Project-Domain-Name: Default Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Project-Id: 03b1f5372fb94c0fb009656de2a99622 Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Project-Name: admin Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Role: member,reader,admin Oct 24 08:44:33.266062 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Roles: member,reader,admin Oct 24 08:44:33.267840 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/03b1f5372fb94c0fb009656de2a99622"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_03b1f5372fb94c0fb009656de2a99622", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/03b1f5372fb94c0fb009656de2a99622"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/03b1f5372fb94c0fb009656de2a99622"}]}] Oct 24 08:44:33.267840 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Tenant: admin Oct 24 08:44:33.267840 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Tenant-Id: 03b1f5372fb94c0fb009656de2a99622 Oct 24 08:44:33.267840 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Tenant-Name: admin Oct 24 08:44:33.267840 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-User: admin Oct 24 08:44:33.267840 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-User-Domain-Id: default Oct 24 08:44:33.267840 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-User-Domain-Name: Default Oct 24 08:44:33.267840 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-User-Id: 0bb4ebf18f65413bbe4945aec1caead8 Oct 24 08:44:33.267840 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-User-Name: admin Oct 24 08:44:33.267840 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: method: > Oct 24 08:44:33.267840 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: exception: Flavor 42 could not be found. Oct 24 08:44:33.267840 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: action_args: {'id': '42'} Oct 24 08:44:33.267840 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:44:33.267840 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:44:33.269206 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Oct 24 08:44:33.270818 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-0bc89a0d-67f3-4c7e-91f4-4ffc7adeb7fd admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=115370) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:44:33.272152 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-0bc89a0d-67f3-4c7e-91f4-4ffc7adeb7fd admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.415194 Oct 24 08:44:33.272152 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 19/39] 10.222.0.39 () {60 vars in 1190 bytes} [Mon Oct 24 08:44:32 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 417 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 24 08:44:33.277707 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-d039f244-cdf0-4ba6-b3ff-a8c292109cb3 admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:33.281776 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG dbcounter [-] [115369] Writing DB stats nova_api:INSERT=1,nova_api:SELECT=2 {{(pid=115369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:44:33.302705 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-d039f244-cdf0-4ba6-b3ff-a8c292109cb3 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.028638 Oct 24 08:44:33.303330 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 21/40] 10.222.0.39 () {60 vars in 1210 bytes} [Mon Oct 24 08:44:33 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:44:33.314088 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-7d5aeb64-73b5-40f8-ade2-1ee59e7fe098 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:33.337835 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-7d5aeb64-73b5-40f8-ade2-1ee59e7fe098 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.028133 Oct 24 08:44:33.338656 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 20/41] 10.222.0.39 () {60 vars in 1210 bytes} [Mon Oct 24 08:44:33 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:44:33.348479 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-90637c3a-7534-4008-93f5-e97957c441f4 admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:33.353803 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.wsgi [None req-90637c3a-7534-4008-93f5-e97957c441f4 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-90637c3a-7534-4008-93f5-e97957c441f4 admin admin] Request method failure captured: Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Accept: application/json Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Accept-Encoding: gzip, deflate Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Connection: keep-alive Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Content-Length: 0 Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Host: 10.222.0.39 Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Openstack-Api-Version: compute 2.92 Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Openstack-System-Scope: None Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: User-Agent: python-novaclient Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Auth-Token: gAAAAABjVlBwYOH1-QFpRRXmsHsQ4meUwJwvOo9I91tpjIcmTi3adsee4OnbL6YpHMlxkaRK7luHGNlQVnnwVbnGO201BQX6Ikm3KWFMoD676WkRE4r2YWSVCjuXC_skgIhMUrG4tJSDAZ4nX-nLqh0SC1f4f9aWnXrryU61MA6EdWrewliDuBQ Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Domain-Id: None Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Domain-Name: None Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Identity-Status: Confirmed Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Is-Admin-Project: True Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Openstack-Nova-Api-Version: 2.92 Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Project-Domain-Id: default Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Project-Domain-Name: Default Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Project-Id: 03b1f5372fb94c0fb009656de2a99622 Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Project-Name: admin Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Role: member,reader,admin Oct 24 08:44:33.357057 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Roles: member,reader,admin Oct 24 08:44:33.358671 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/03b1f5372fb94c0fb009656de2a99622"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_03b1f5372fb94c0fb009656de2a99622", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/03b1f5372fb94c0fb009656de2a99622"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/03b1f5372fb94c0fb009656de2a99622"}]}] Oct 24 08:44:33.358671 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Tenant: admin Oct 24 08:44:33.358671 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Tenant-Id: 03b1f5372fb94c0fb009656de2a99622 Oct 24 08:44:33.358671 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Tenant-Name: admin Oct 24 08:44:33.358671 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-User: admin Oct 24 08:44:33.358671 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-User-Domain-Id: default Oct 24 08:44:33.358671 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-User-Domain-Name: Default Oct 24 08:44:33.358671 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-User-Id: 0bb4ebf18f65413bbe4945aec1caead8 Oct 24 08:44:33.358671 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-User-Name: admin Oct 24 08:44:33.358671 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: method: > Oct 24 08:44:33.358671 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: exception: Flavor 42 could not be found. Oct 24 08:44:33.358671 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: action_args: {'id': '42'} Oct 24 08:44:33.358671 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:44:33.358671 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:44:33.359734 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Oct 24 08:44:33.361208 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-90637c3a-7534-4008-93f5-e97957c441f4 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=115369) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:44:33.364116 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-90637c3a-7534-4008-93f5-e97957c441f4 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.013957 Oct 24 08:44:33.364116 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 22/42] 10.222.0.39 () {60 vars in 1190 bytes} [Mon Oct 24 08:44:33 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 24 08:44:34.604122 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-7d5aeb64-73b5-40f8-ade2-1ee59e7fe098 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:34.605165 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-7d5aeb64-73b5-40f8-ade2-1ee59e7fe098 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001472 Oct 24 08:44:34.605568 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 21/43] 10.222.0.39 () {56 vars in 1074 bytes} [Mon Oct 24 08:44:34 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:44:35.161260 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-8a78150d-d33b-4e5b-90d6-166c9b111983 admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:35.166308 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.wsgi [None req-8a78150d-d33b-4e5b-90d6-166c9b111983 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-8a78150d-d33b-4e5b-90d6-166c9b111983 admin admin] Request method failure captured: Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Accept: application/json Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Accept-Encoding: gzip, deflate Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Connection: keep-alive Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Content-Length: 0 Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Host: 10.222.0.39 Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Openstack-Api-Version: compute 2.92 Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Openstack-System-Scope: None Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: User-Agent: python-novaclient Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Auth-Token: gAAAAABjVlByvy0eqIjJy0fvmB61BLOBUbPQMt-Htu1yfbDqvyU0GyWt3YORgrO5__Kd6SzmhwTyQ2e-Ez4oba7VQoatQPi_PZSLfb5b0RLEjeguCPyiU0ohGERAsA-Yvrn9WhsbCZXbabbOPF_R3yAK5RqcmKsPqrKVjUr-Pq49dsL3b9mfI5o Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Domain-Id: None Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Domain-Name: None Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Identity-Status: Confirmed Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Is-Admin-Project: True Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Openstack-Nova-Api-Version: 2.92 Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Project-Domain-Id: default Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Project-Domain-Name: Default Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Project-Id: 03b1f5372fb94c0fb009656de2a99622 Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Project-Name: admin Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Role: member,reader,admin Oct 24 08:44:35.167821 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Roles: member,reader,admin Oct 24 08:44:35.169536 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/03b1f5372fb94c0fb009656de2a99622"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_03b1f5372fb94c0fb009656de2a99622", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/03b1f5372fb94c0fb009656de2a99622"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/03b1f5372fb94c0fb009656de2a99622"}]}] Oct 24 08:44:35.169536 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Tenant: admin Oct 24 08:44:35.169536 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Tenant-Id: 03b1f5372fb94c0fb009656de2a99622 Oct 24 08:44:35.169536 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Tenant-Name: admin Oct 24 08:44:35.169536 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-User: admin Oct 24 08:44:35.169536 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-User-Domain-Id: default Oct 24 08:44:35.169536 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-User-Domain-Name: Default Oct 24 08:44:35.169536 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-User-Id: 0bb4ebf18f65413bbe4945aec1caead8 Oct 24 08:44:35.169536 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-User-Name: admin Oct 24 08:44:35.169536 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: method: > Oct 24 08:44:35.169536 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: exception: Flavor 84 could not be found. Oct 24 08:44:35.169536 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: action_args: {'id': '84'} Oct 24 08:44:35.169536 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:44:35.169536 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:44:35.170529 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Oct 24 08:44:35.171640 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-8a78150d-d33b-4e5b-90d6-166c9b111983 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=115369) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:44:35.172684 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-8a78150d-d33b-4e5b-90d6-166c9b111983 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.420557 Oct 24 08:44:35.172684 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 23/44] 10.222.0.39 () {60 vars in 1190 bytes} [Mon Oct 24 08:44:34 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 422 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 24 08:44:35.179489 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-45424e11-4130-40cc-87df-40485bb34192 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:35.195604 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-45424e11-4130-40cc-87df-40485bb34192 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019687 Oct 24 08:44:35.196192 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 22/45] 10.222.0.39 () {60 vars in 1210 bytes} [Mon Oct 24 08:44:35 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:44:35.206606 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-2d481584-ad69-4640-808c-1eb7db0c1c8d admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:35.222071 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-2d481584-ad69-4640-808c-1eb7db0c1c8d admin admin] 10.222.0.39 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019602 Oct 24 08:44:35.222619 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 24/46] 10.222.0.39 () {60 vars in 1210 bytes} [Mon Oct 24 08:44:35 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:44:35.232292 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-d10dfb74-3d2d-403b-a0f7-573b1f569844 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:35.237284 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.wsgi [None req-d10dfb74-3d2d-403b-a0f7-573b1f569844 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-d10dfb74-3d2d-403b-a0f7-573b1f569844 admin admin] Request method failure captured: Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Accept: application/json Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Accept-Encoding: gzip, deflate Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Connection: keep-alive Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Content-Length: 0 Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Host: 10.222.0.39 Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Openstack-Api-Version: compute 2.92 Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Openstack-System-Scope: None Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: User-Agent: python-novaclient Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Auth-Token: gAAAAABjVlByvy0eqIjJy0fvmB61BLOBUbPQMt-Htu1yfbDqvyU0GyWt3YORgrO5__Kd6SzmhwTyQ2e-Ez4oba7VQoatQPi_PZSLfb5b0RLEjeguCPyiU0ohGERAsA-Yvrn9WhsbCZXbabbOPF_R3yAK5RqcmKsPqrKVjUr-Pq49dsL3b9mfI5o Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Domain-Id: None Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Domain-Name: None Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Identity-Status: Confirmed Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Is-Admin-Project: True Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Openstack-Nova-Api-Version: 2.92 Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Project-Domain-Id: default Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Project-Domain-Name: Default Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Project-Id: 03b1f5372fb94c0fb009656de2a99622 Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Project-Name: admin Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Role: member,reader,admin Oct 24 08:44:35.238783 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Roles: member,reader,admin Oct 24 08:44:35.240418 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/03b1f5372fb94c0fb009656de2a99622"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_03b1f5372fb94c0fb009656de2a99622", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/03b1f5372fb94c0fb009656de2a99622"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/03b1f5372fb94c0fb009656de2a99622"}]}] Oct 24 08:44:35.240418 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Tenant: admin Oct 24 08:44:35.240418 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Tenant-Id: 03b1f5372fb94c0fb009656de2a99622 Oct 24 08:44:35.240418 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Tenant-Name: admin Oct 24 08:44:35.240418 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-User: admin Oct 24 08:44:35.240418 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-User-Domain-Id: default Oct 24 08:44:35.240418 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-User-Domain-Name: Default Oct 24 08:44:35.240418 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-User-Id: 0bb4ebf18f65413bbe4945aec1caead8 Oct 24 08:44:35.240418 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-User-Name: admin Oct 24 08:44:35.240418 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: method: > Oct 24 08:44:35.240418 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: exception: Flavor 84 could not be found. Oct 24 08:44:35.240418 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: action_args: {'id': '84'} Oct 24 08:44:35.240418 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:44:35.240418 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:44:35.241386 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Oct 24 08:44:35.242515 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-d10dfb74-3d2d-403b-a0f7-573b1f569844 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=115370) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:44:35.243597 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-d10dfb74-3d2d-403b-a0f7-573b1f569844 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011523 Oct 24 08:44:35.243597 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 23/47] 10.222.0.39 () {60 vars in 1190 bytes} [Mon Oct 24 08:44:35 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 24 08:44:36.493793 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-2d481584-ad69-4640-808c-1eb7db0c1c8d admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:36.494825 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-2d481584-ad69-4640-808c-1eb7db0c1c8d admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001497 Oct 24 08:44:36.495226 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 25/48] 10.222.0.39 () {56 vars in 1074 bytes} [Mon Oct 24 08:44:36 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:44:37.052638 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-a69247b2-498e-4af4-93d8-1f716b2bd3a4 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:37.058553 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.wsgi [None req-a69247b2-498e-4af4-93d8-1f716b2bd3a4 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-a69247b2-498e-4af4-93d8-1f716b2bd3a4 admin admin] Request method failure captured: Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Accept: application/json Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Accept-Encoding: gzip, deflate Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Connection: keep-alive Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Content-Length: 0 Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Host: 10.222.0.39 Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Openstack-Api-Version: compute 2.92 Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Openstack-System-Scope: None Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: User-Agent: python-novaclient Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Auth-Token: gAAAAABjVlB0xDXBCtXAYsi9clydeCIyO6si-Hhya3OEsIr1yeYQyexUciVfO-3dQSd6wf2RJmm3Dg8fcELwvESA-JCL1hcIRTTyB0o3TUtDKnpsz54mPEp20NnK50l3HiCvYLxTJYw3-3QpKIuVRJ04xnzCKni2Hruh4VU3Kx9D0opB9U2GhPY Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Domain-Id: None Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Domain-Name: None Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Identity-Status: Confirmed Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Is-Admin-Project: True Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Openstack-Nova-Api-Version: 2.92 Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Project-Domain-Id: default Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Project-Domain-Name: Default Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Project-Id: 03b1f5372fb94c0fb009656de2a99622 Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Project-Name: admin Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Role: member,reader,admin Oct 24 08:44:37.060059 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Roles: member,reader,admin Oct 24 08:44:37.061730 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/03b1f5372fb94c0fb009656de2a99622"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_03b1f5372fb94c0fb009656de2a99622", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/03b1f5372fb94c0fb009656de2a99622"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/03b1f5372fb94c0fb009656de2a99622"}]}] Oct 24 08:44:37.061730 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Tenant: admin Oct 24 08:44:37.061730 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Tenant-Id: 03b1f5372fb94c0fb009656de2a99622 Oct 24 08:44:37.061730 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-Tenant-Name: admin Oct 24 08:44:37.061730 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-User: admin Oct 24 08:44:37.061730 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-User-Domain-Id: default Oct 24 08:44:37.061730 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-User-Domain-Name: Default Oct 24 08:44:37.061730 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-User-Id: 0bb4ebf18f65413bbe4945aec1caead8 Oct 24 08:44:37.061730 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: X-User-Name: admin Oct 24 08:44:37.061730 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: method: > Oct 24 08:44:37.061730 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: exception: Flavor 451 could not be found. Oct 24 08:44:37.061730 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: action_args: {'id': '451'} Oct 24 08:44:37.061730 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:44:37.061730 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:44:37.062727 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: ERROR nova.api.openstack.wsgi Oct 24 08:44:37.064107 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-a69247b2-498e-4af4-93d8-1f716b2bd3a4 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=115370) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:44:37.065513 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-a69247b2-498e-4af4-93d8-1f716b2bd3a4 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.425055 Oct 24 08:44:37.065513 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 24/49] 10.222.0.39 () {60 vars in 1193 bytes} [Mon Oct 24 08:44:36 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 426 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 24 08:44:37.071575 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-8b921280-11f6-428a-af3e-13243902c89b admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:37.088359 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-8b921280-11f6-428a-af3e-13243902c89b admin admin] 10.222.0.39 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.020296 Oct 24 08:44:37.088926 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 26/50] 10.222.0.39 () {60 vars in 1210 bytes} [Mon Oct 24 08:44:37 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:44:37.099754 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-b03ef574-714f-407b-99b9-3c9418185b83 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:37.115169 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-b03ef574-714f-407b-99b9-3c9418185b83 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019772 Oct 24 08:44:37.115586 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 25/51] 10.222.0.39 () {60 vars in 1210 bytes} [Mon Oct 24 08:44:37 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:44:37.125406 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-48609823-e429-4281-a4ac-6b3eec950230 admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:37.130980 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.wsgi [None req-48609823-e429-4281-a4ac-6b3eec950230 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-48609823-e429-4281-a4ac-6b3eec950230 admin admin] Request method failure captured: Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Accept: application/json Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Accept-Encoding: gzip, deflate Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Connection: keep-alive Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Content-Length: 0 Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Host: 10.222.0.39 Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Openstack-Api-Version: compute 2.92 Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Openstack-System-Scope: None Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: User-Agent: python-novaclient Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Auth-Token: gAAAAABjVlB0xDXBCtXAYsi9clydeCIyO6si-Hhya3OEsIr1yeYQyexUciVfO-3dQSd6wf2RJmm3Dg8fcELwvESA-JCL1hcIRTTyB0o3TUtDKnpsz54mPEp20NnK50l3HiCvYLxTJYw3-3QpKIuVRJ04xnzCKni2Hruh4VU3Kx9D0opB9U2GhPY Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Domain-Id: None Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Domain-Name: None Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Identity-Status: Confirmed Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Is-Admin-Project: True Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Openstack-Nova-Api-Version: 2.92 Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Project-Domain-Id: default Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Project-Domain-Name: Default Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Project-Id: 03b1f5372fb94c0fb009656de2a99622 Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Project-Name: admin Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Role: member,reader,admin Oct 24 08:44:37.132527 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Roles: member,reader,admin Oct 24 08:44:37.134075 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/03b1f5372fb94c0fb009656de2a99622"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_03b1f5372fb94c0fb009656de2a99622", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/03b1f5372fb94c0fb009656de2a99622"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/03b1f5372fb94c0fb009656de2a99622"}]}] Oct 24 08:44:37.134075 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Tenant: admin Oct 24 08:44:37.134075 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Tenant-Id: 03b1f5372fb94c0fb009656de2a99622 Oct 24 08:44:37.134075 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-Tenant-Name: admin Oct 24 08:44:37.134075 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-User: admin Oct 24 08:44:37.134075 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-User-Domain-Id: default Oct 24 08:44:37.134075 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-User-Domain-Name: Default Oct 24 08:44:37.134075 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-User-Id: 0bb4ebf18f65413bbe4945aec1caead8 Oct 24 08:44:37.134075 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: X-User-Name: admin Oct 24 08:44:37.134075 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: method: > Oct 24 08:44:37.134075 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: exception: Flavor 451 could not be found. Oct 24 08:44:37.134075 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: action_args: {'id': '451'} Oct 24 08:44:37.134075 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:44:37.134075 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:44:37.135161 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: ERROR nova.api.openstack.wsgi Oct 24 08:44:37.136663 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-48609823-e429-4281-a4ac-6b3eec950230 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=115369) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:44:37.138015 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-48609823-e429-4281-a4ac-6b3eec950230 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.012400 Oct 24 08:44:37.138015 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 27/52] 10.222.0.39 () {60 vars in 1193 bytes} [Mon Oct 24 08:44:37 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Oct 24 08:44:38.381702 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-b03ef574-714f-407b-99b9-3c9418185b83 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:38.382720 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-b03ef574-714f-407b-99b9-3c9418185b83 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001459 Oct 24 08:44:38.383118 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 26/53] 10.222.0.39 () {56 vars in 1074 bytes} [Mon Oct 24 08:44:38 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:44:38.926763 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-acd0781a-c19b-4fbe-be2b-8357ea113a8a admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:44:38.946953 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-acd0781a-c19b-4fbe-be2b-8357ea113a8a admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.422935 Oct 24 08:44:38.947491 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 28/54] 10.222.0.39 () {64 vars in 1235 bytes} [Mon Oct 24 08:44:38 2022] POST /compute/v2.1/flavors => generated 404 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:44:40.180526 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-b03ef574-714f-407b-99b9-3c9418185b83 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:40.181572 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-b03ef574-714f-407b-99b9-3c9418185b83 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001507 Oct 24 08:44:40.181978 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 27/55] 10.222.0.39 () {56 vars in 1074 bytes} [Mon Oct 24 08:44:40 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:44:40.713483 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-f2d4d618-7fca-4a79-a0cc-c9bef558fead admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:44:40.732549 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-f2d4d618-7fca-4a79-a0cc-c9bef558fead admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.411427 Oct 24 08:44:40.733094 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 29/56] 10.222.0.39 () {64 vars in 1235 bytes} [Mon Oct 24 08:44:40 2022] POST /compute/v2.1/flavors => generated 405 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:44:41.964225 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-b03ef574-714f-407b-99b9-3c9418185b83 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:41.965240 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-b03ef574-714f-407b-99b9-3c9418185b83 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001508 Oct 24 08:44:41.965641 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 28/57] 10.222.0.39 () {56 vars in 1074 bytes} [Mon Oct 24 08:44:41 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:44:42.512305 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-c5322fad-9d45-4053-9527-910312503e86 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:44:42.531607 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-c5322fad-9d45-4053-9527-910312503e86 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.426423 Oct 24 08:44:42.532196 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 30/58] 10.222.0.39 () {64 vars in 1235 bytes} [Mon Oct 24 08:44:42 2022] POST /compute/v2.1/flavors => generated 407 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:44:43.767646 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-b03ef574-714f-407b-99b9-3c9418185b83 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:43.769297 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-b03ef574-714f-407b-99b9-3c9418185b83 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002567 Oct 24 08:44:43.769690 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 29/59] 10.222.0.39 () {56 vars in 1074 bytes} [Mon Oct 24 08:44:43 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:44:44.329456 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-4498b24a-fc54-4f79-8a1d-13df43d70486 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:44:44.348876 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-4498b24a-fc54-4f79-8a1d-13df43d70486 admin admin] 10.222.0.39 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.431942 Oct 24 08:44:44.349430 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 31/60] 10.222.0.39 () {64 vars in 1235 bytes} [Mon Oct 24 08:44:43 2022] POST /compute/v2.1/flavors => generated 409 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:44:45.578885 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-b03ef574-714f-407b-99b9-3c9418185b83 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:45.580252 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-b03ef574-714f-407b-99b9-3c9418185b83 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001487 Oct 24 08:44:45.580656 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 30/61] 10.222.0.39 () {56 vars in 1074 bytes} [Mon Oct 24 08:44:45 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:44:46.122254 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-9fd56231-044b-4120-953d-983d0dbe9e0d admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:44:46.144028 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-9fd56231-044b-4120-953d-983d0dbe9e0d admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.421429 Oct 24 08:44:46.144591 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 32/62] 10.222.0.39 () {60 vars in 1202 bytes} [Mon Oct 24 08:44:45 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:45:44.599383 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-b03ef574-714f-407b-99b9-3c9418185b83 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:45:44.600362 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-b03ef574-714f-407b-99b9-3c9418185b83 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001455 Oct 24 08:45:44.600764 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 31/63] 10.222.0.39 () {54 vars in 931 bytes} [Mon Oct 24 08:45:44 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:45:44.611616 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG dbcounter [-] [115369] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=115369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:45:45.005074 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-be4ba4b2-f7b3-4bcd-8a97-49529e3f7111 admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:45:45.026923 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-be4ba4b2-f7b3-4bcd-8a97-49529e3f7111 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.420063 Oct 24 08:45:45.027462 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 33/64] 10.222.0.39 () {60 vars in 1292 bytes} [Mon Oct 24 08:45:44 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:45:45.050176 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG dbcounter [-] [115370] Writing DB stats nova_api:SELECT=14 {{(pid=115370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:45:45.054121 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-a26021ec-4f5c-4de6-b244-f62b171ca569 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:45:45.062128 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-a26021ec-4f5c-4de6-b244-f62b171ca569 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013325 Oct 24 08:45:45.062658 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 32/65] 10.222.0.39 () {60 vars in 1283 bytes} [Mon Oct 24 08:45:45 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 15 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:45:45.074730 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-dea9138d-a8d6-49f2-b703-d0ee9d03b16b admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:45:45.083429 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-dea9138d-a8d6-49f2-b703-d0ee9d03b16b admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012353 Oct 24 08:45:45.083966 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 34/66] 10.222.0.39 () {60 vars in 1286 bytes} [Mon Oct 24 08:45:45 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:45:45.096912 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-a3f85ebc-1e3a-408d-9fff-aa0ce8c74790 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:45:45.104538 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-a3f85ebc-1e3a-408d-9fff-aa0ce8c74790 admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011196 Oct 24 08:45:45.105077 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 33/67] 10.222.0.39 () {60 vars in 1286 bytes} [Mon Oct 24 08:45:45 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:45:45.118077 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-54c68909-f3fb-4ea8-a709-906f002417bc admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:45:45.126692 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-54c68909-f3fb-4ea8-a709-906f002417bc admin admin] 10.222.0.39 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012261 Oct 24 08:45:45.127241 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 35/68] 10.222.0.39 () {60 vars in 1283 bytes} [Mon Oct 24 08:45:45 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:48:46.575422 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-a3f85ebc-1e3a-408d-9fff-aa0ce8c74790 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:48:46.576462 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-a3f85ebc-1e3a-408d-9fff-aa0ce8c74790 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001456 Oct 24 08:48:46.576859 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 34/69] 10.222.0.39 () {56 vars in 1074 bytes} [Mon Oct 24 08:48:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:48:46.728877 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG dbcounter [-] [115369] Writing DB stats nova_api:SELECT=6 {{(pid=115369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:46.892898 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-a57ac853-eb67-4765-a5ea-96951182093c admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:48:46.897833 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-a57ac853-eb67-4765-a5ea-96951182093c admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:48:46.898308 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-a57ac853-eb67-4765-a5ea-96951182093c admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:48:46.898733 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-a57ac853-eb67-4765-a5ea-96951182093c admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:48:46.899439 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-a57ac853-eb67-4765-a5ea-96951182093c admin admin] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:48:46.899885 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-a57ac853-eb67-4765-a5ea-96951182093c admin admin] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:48:46.900341 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-a57ac853-eb67-4765-a5ea-96951182093c admin admin] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:48:46.932521 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-a57ac853-eb67-4765-a5ea-96951182093c admin admin] 10.222.0.39 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.204813 Oct 24 08:48:46.933103 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 36/70] 10.222.0.39 () {60 vars in 1193 bytes} [Mon Oct 24 08:48:46 2022] GET /compute/v2.1/os-services => generated 791 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:48:48.064021 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-a3f85ebc-1e3a-408d-9fff-aa0ce8c74790 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:48:48.065082 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-a3f85ebc-1e3a-408d-9fff-aa0ce8c74790 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001495 Oct 24 08:48:48.065482 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 35/71] 10.222.0.39 () {56 vars in 1074 bytes} [Mon Oct 24 08:48:48 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:48:48.272114 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-618712c3-112e-4cc5-ac7f-2d7bbad5f032 admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:48:48.275637 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-618712c3-112e-4cc5-ac7f-2d7bbad5f032 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:48:48.276175 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-618712c3-112e-4cc5-ac7f-2d7bbad5f032 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:48:48.276612 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-618712c3-112e-4cc5-ac7f-2d7bbad5f032 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:48:48.277811 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-618712c3-112e-4cc5-ac7f-2d7bbad5f032 admin admin] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:48:48.277811 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-618712c3-112e-4cc5-ac7f-2d7bbad5f032 admin admin] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:48:48.278297 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-618712c3-112e-4cc5-ac7f-2d7bbad5f032 admin admin] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:48:48.302332 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-618712c3-112e-4cc5-ac7f-2d7bbad5f032 admin admin] 10.222.0.39 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.092021 Oct 24 08:48:48.302918 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 37/72] 10.222.0.39 () {60 vars in 1193 bytes} [Mon Oct 24 08:48:48 2022] GET /compute/v2.1/os-services => generated 791 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:48:59.821267 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-a3f85ebc-1e3a-408d-9fff-aa0ce8c74790 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:48:59.822457 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-a3f85ebc-1e3a-408d-9fff-aa0ce8c74790 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001733 Oct 24 08:48:59.822918 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 36/73] 10.222.0.39 () {56 vars in 1074 bytes} [Mon Oct 24 08:48:59 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:48:59.973487 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG dbcounter [-] [115369] Writing DB stats nova_cell0:SELECT=4 {{(pid=115369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:59.974237 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG dbcounter [-] [115369] Writing DB stats nova_cell1:SELECT=4 {{(pid=115369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:59.974858 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG dbcounter [-] [115369] Writing DB stats nova_api:SELECT=8 {{(pid=115369) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:00.028458 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-d2d1602d-df9d-452f-82f5-4a1e64463807 admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:49:00.031752 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-d2d1602d-df9d-452f-82f5-4a1e64463807 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:49:00.032252 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-d2d1602d-df9d-452f-82f5-4a1e64463807 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:49:00.032673 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-d2d1602d-df9d-452f-82f5-4a1e64463807 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:49:00.033376 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-d2d1602d-df9d-452f-82f5-4a1e64463807 admin admin] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:49:00.033877 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-d2d1602d-df9d-452f-82f5-4a1e64463807 admin admin] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:49:00.034222 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-d2d1602d-df9d-452f-82f5-4a1e64463807 admin admin] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:49:00.060440 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-d2d1602d-df9d-452f-82f5-4a1e64463807 admin admin] 10.222.0.39 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.091427 Oct 24 08:49:00.060991 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 38/74] 10.222.0.39 () {60 vars in 1193 bytes} [Mon Oct 24 08:48:59 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:49:01.159125 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG nova.api.openstack.wsgi [None req-a3f85ebc-1e3a-408d-9fff-aa0ce8c74790 admin admin] Calling method '>' {{(pid=115370) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:49:01.160185 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: INFO nova.api.openstack.requestlog [None req-a3f85ebc-1e3a-408d-9fff-aa0ce8c74790 admin admin] 10.222.0.39 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001634 Oct 24 08:49:01.160554 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: [pid: 115370|app: 0|req: 37/75] 10.222.0.39 () {56 vars in 1074 bytes} [Mon Oct 24 08:49:01 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:49:01.353827 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG nova.api.openstack.wsgi [None req-27fc0ab3-7833-4ff7-8f8d-f9caffeea53a admin admin] Calling method '>' {{(pid=115369) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:49:01.357398 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-27fc0ab3-7833-4ff7-8f8d-f9caffeea53a admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:49:01.357872 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-27fc0ab3-7833-4ff7-8f8d-f9caffeea53a admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:49:01.358299 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-27fc0ab3-7833-4ff7-8f8d-f9caffeea53a admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:49:01.359010 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-27fc0ab3-7833-4ff7-8f8d-f9caffeea53a admin admin] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:49:01.359450 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-27fc0ab3-7833-4ff7-8f8d-f9caffeea53a admin admin] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:49:01.359862 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: DEBUG oslo_concurrency.lockutils [None req-27fc0ab3-7833-4ff7-8f8d-f9caffeea53a admin admin] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=115369) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:49:01.381929 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: INFO nova.api.openstack.requestlog [None req-27fc0ab3-7833-4ff7-8f8d-f9caffeea53a admin admin] 10.222.0.39 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.080405 Oct 24 08:49:01.382491 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: [pid: 115369|app: 0|req: 39/76] 10.222.0.39 () {60 vars in 1193 bytes} [Mon Oct 24 08:49:01 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:49:59.708525 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: Mon Oct 24 08:49:59 2022 - graceful shutdown triggered... Oct 24 08:49:59.708661 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Oct 24 08:49:59.709675 dv-cinder-master-3-iscsi devstack@n-api.service[115369]: Gracefully killing worker 1 (pid: 115369)... Oct 24 08:49:59.710128 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: Gracefully killing worker 2 (pid: 115370)... Oct 24 08:49:59.840220 dv-cinder-master-3-iscsi devstack@n-api.service[115370]: DEBUG dbcounter [-] [115370] Writing DB stats nova_api:SELECT=4 {{(pid=115370) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:00.709994 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: worker 1 buried after 1 seconds Oct 24 08:50:00.710584 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: worker 2 buried after 1 seconds Oct 24 08:50:00.710584 dv-cinder-master-3-iscsi devstack@n-api.service[115368]: goodbye to uWSGI. Oct 24 08:50:00.711636 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Oct 24 08:50:00.712271 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Oct 24 08:50:00.715099 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 24 08:50:00.726100 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 24 08:50:00.728174 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 24 08:50:00 2022] *** Oct 24 08:50:00.728174 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 24 08:50:00.728174 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 24 08:50:00.728174 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: nodename: dv-cinder-master-3-iscsi Oct 24 08:50:00.728174 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: machine: x86_64 Oct 24 08:50:00.728174 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: clock source: unix Oct 24 08:50:00.728174 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: pcre jit disabled Oct 24 08:50:00.728174 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: detected number of CPU cores: 4 Oct 24 08:50:00.728174 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: current working directory: / Oct 24 08:50:00.728174 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: detected binary path: /usr/bin/uwsgi-core Oct 24 08:50:00.728959 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: your processes number limit is 47680 Oct 24 08:50:00.728959 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: your memory page size is 4096 bytes Oct 24 08:50:00.728959 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: detected max file descriptor number: 2048 Oct 24 08:50:00.728959 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: lock engine: pthread robust mutexes Oct 24 08:50:00.728959 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: thunder lock: enabled Oct 24 08:50:00.728959 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 24 08:50:00.728959 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 24 08:50:00.769776 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: Python main interpreter initialized at 0x55c51bf019c0 Oct 24 08:50:00.769776 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: python threads support enabled Oct 24 08:50:00.769776 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: your server socket listen backlog is limited to 100 connections Oct 24 08:50:00.769776 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: your mercy for graceful operations on workers is 90 seconds Oct 24 08:50:00.770199 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: mapped 403077 bytes (393 KB) for 2 cores Oct 24 08:50:00.770311 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: *** Operational MODE: preforking *** Oct 24 08:50:00.770636 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: *** uWSGI is running in multiple interpreter mode *** Oct 24 08:50:00.770636 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: spawned uWSGI master process (pid: 130377) Oct 24 08:50:00.770847 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 24 08:50:00.779156 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: spawned uWSGI worker 1 (pid: 130395, cores: 1) Oct 24 08:50:00.779156 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: spawned uWSGI worker 2 (pid: 130396, cores: 1) Oct 24 08:50:00.779156 dv-cinder-master-3-iscsi devstack@n-api.service[130377]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 24 08:50:01.450797 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 24 08:50:01.459283 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 24 08:50:02.713668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 08:50:02.713668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: from cryptography.utils import int_from_bytes Oct 24 08:50:02.714494 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 08:50:02.714494 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: from cryptography.utils import int_from_bytes Oct 24 08:50:02.733667 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 08:50:02.733667 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: from cryptography.utils import int_from_bytes Oct 24 08:50:02.734194 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 08:50:02.734194 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: from cryptography.utils import int_from_bytes Oct 24 08:50:05.192177 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 24 08:50:05.192811 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 24 08:50:05.192973 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 24 08:50:05.193345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 24 08:50:05.193608 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 24 08:50:05.193997 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.194326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.194647 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.194967 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.195299 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.195613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.195937 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.196282 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.197605 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.197605 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.197605 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.197605 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.197925 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.198123 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.198409 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.198731 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.199037 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.199362 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.199668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.200197 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.201617 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.201617 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.201617 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.201617 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.201958 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.202188 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.202513 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.202832 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.203153 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.203466 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.203881 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.204249 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.204550 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.204866 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.205296 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.205622 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.205929 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.206256 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.206631 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.206972 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.207272 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.207601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.207927 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.208276 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.208587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.208900 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.209215 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.209539 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.210031 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.210483 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.210891 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.211368 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.211852 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.212345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.212848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.213272 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.213691 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.214204 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.214535 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.215225 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.215225 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.215584 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.215764 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.216077 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.216415 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.216728 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.217039 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.217365 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.217686 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.218014 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.218325 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.218650 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.219064 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.39 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.219317 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.39 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.219633 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.219988 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.220358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.220674 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.220997 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.221322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.221634 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.221980 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.222504 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.222832 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.223187 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.223496 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.223753 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.224069 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.224402 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.224728 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.225043 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.225356 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.225663 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.225964 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.226278 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.226589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.226894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.227202 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.227510 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.227820 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.228145 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.228456 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.228766 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.229074 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.229396 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.229536 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 24 08:50:05.230015 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.230123 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 24 08:50:05.230239 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.230378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.230480 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 24 08:50:05.230624 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.230749 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 24 08:50:05.230922 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.231029 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 24 08:50:05.231245 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.231417 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.231589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.231751 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.231897 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.232042 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.232286 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.232437 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.232591 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.232751 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.232898 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.233043 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.233197 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.233371 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.233545 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.233700 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.233852 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.234002 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.234148 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.234304 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.234460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.234604 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.234750 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.234904 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.235057 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.235211 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.235356 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.235519 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.235665 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.235811 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.235956 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.236114 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.236309 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.236464 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.236609 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.236762 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.236907 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.237067 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.237218 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.237363 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.237516 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.238248 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.238412 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.238564 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.238720 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.238867 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.239018 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.239163 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.239317 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.239467 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.239613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.239788 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.240034 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.240203 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.240467 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.240731 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.240894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.241051 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.241197 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.241360 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.241767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.241920 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.242075 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.242228 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.242397 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.242556 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.242702 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.242847 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.242999 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.243144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.243298 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.243448 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.243601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.243746 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.243902 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.244048 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.244356 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.244509 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.244654 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.244808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.244960 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.245119 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.245266 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.245419 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.245565 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.245717 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.245863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.246022 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.246168 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.246335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.246481 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.246637 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.246789 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.246949 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.247095 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.247240 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.247394 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.247547 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.247702 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.247862 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.248007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.248193 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.248355 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.248510 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.248669 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.248821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.248968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.249114 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.249259 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.249406 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.249559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.249712 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.249858 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.250035 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.250274 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.250274 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.250577 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.250753 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.250979 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.251128 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.251296 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.251452 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.251615 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.251761 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.251906 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.252072 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.252252 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.252409 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.252564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.252709 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.252862 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.253018 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.253190 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.253982 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.254132 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.254277 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.254432 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.254651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.254905 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.255067 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.255250 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.255406 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.255559 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.255717 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.255717 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.256082 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.256082 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.256389 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.39 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.256576 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.256739 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.39 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.256899 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.257057 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.257212 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.257374 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.257521 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.257673 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.257859 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.258015 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.258161 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.258306 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.258451 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.258603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.258758 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.258905 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.259070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.259223 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.259374 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.259534 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.259687 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.259832 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.259980 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.260220 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.260481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.260649 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.260805 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.260971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.261196 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.261378 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.261538 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.261694 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.261848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.262002 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.262155 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.262308 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.262453 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.262612 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.262759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.262911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.263057 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.263213 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.263368 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.263521 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.263674 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.263828 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.263986 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.264270 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.264434 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.264581 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.264741 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.264887 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.265039 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.265191 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.265354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.265515 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.265662 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.265807 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.265951 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.266104 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.266250 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.266403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.266548 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.266701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.266846 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.266999 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.267144 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.267310 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.267456 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.267601 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.267746 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.267973 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.268142 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.268298 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.268443 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.268596 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.268751 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.268992 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.269227 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.269426 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.269597 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.269758 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.269904 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.270055 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.270212 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.270359 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.270618 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.270787 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.270941 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.271087 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.271336 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.271499 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.271657 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.271812 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.271966 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.272112 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.272333 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.272488 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.272634 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.272789 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.272934 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.273085 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.273231 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.273383 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.273543 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.273703 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.273849 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.274000 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.274145 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.274290 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.274444 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.274591 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.274744 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.274897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.275050 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.275195 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.275354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.275500 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.275703 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.275862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.276016 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.276189 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.276354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.276500 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.276500 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.276775 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.276927 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.277073 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.277226 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:50:05.277372 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.277372 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.277655 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.277810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.277964 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.278117 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.278263 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.278408 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.278553 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.278705 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.278860 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.279006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.279158 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.279306 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.279470 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.279622 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.279777 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.279923 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.280068 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.280253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.280407 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.280564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.280820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.280972 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.281117 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.281279 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.281522 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.281697 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.281842 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.281997 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.282142 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.282286 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.282431 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.282585 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.282737 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.282883 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.283028 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.283173 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.283328 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.283485 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.283640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.283786 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.283938 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.284093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.284285 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.284481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.284642 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.284796 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.284948 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.285095 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.285253 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.285407 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.285552 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.285718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.285871 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.286024 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.286180 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.286333 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.286479 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.286631 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.286778 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.286923 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.287077 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.287232 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.287377 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.287531 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.287677 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.287823 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.287977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.288123 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.288351 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.288510 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.288666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.288827 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.288973 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.289126 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.289273 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.289417 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.289572 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.289728 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.289873 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.290026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.290178 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.290334 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.290479 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.290633 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.290633 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.291030 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.291196 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.291348 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.291513 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.291784 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.291943 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.292097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.292285 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.292433 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.292588 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.292735 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.292887 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.293040 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.293194 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.293348 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.293505 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.293659 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.293831 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.293985 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.294139 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.294284 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.294438 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.294586 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.294739 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.294885 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.295030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.295181 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.295339 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.295493 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.295640 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.295796 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.295943 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.296097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.296296 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.296442 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.296588 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.296588 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.296864 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.297093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.297248 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.297403 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.297548 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.297715 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.297862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.298016 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.298161 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.298314 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.298460 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.298605 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.298750 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.298909 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.299063 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.299216 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.299362 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.299507 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.299677 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.299824 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.299977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.300783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.300949 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.301120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.301328 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.301495 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.301646 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.301794 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.301947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.302107 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.302277 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.302475 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.302647 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.302795 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.302958 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.303117 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.303283 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.303468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.303618 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.303755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.303892 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.304040 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.304261 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.304399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.304878 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.305034 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.305034 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.305524 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.305524 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.305933 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.306094 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.306259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.306616 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.306811 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.307157 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.307306 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.307442 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.307580 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.307739 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.307909 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.308066 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.308249 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.308412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.308558 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.308710 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.308856 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.309006 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.309150 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.309329 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.309513 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.309668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.310026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.310185 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.310534 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.310702 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.310856 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.310856 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.311154 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.311312 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.311484 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.311646 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.311819 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.311968 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.312119 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.312296 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.312449 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.312595 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.312720 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.312866 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.313028 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.313193 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.313359 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.313511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.313655 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.313807 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.313807 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.314091 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.314281 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.314445 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.314592 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.314752 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.314910 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.315051 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.315204 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.315351 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.315508 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.315508 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.316681 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.317591 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.317757 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.317916 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.318069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.318247 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.318417 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.318585 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.318745 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.318904 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.319064 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.319210 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.319210 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.319516 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.319668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.319820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.319976 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.320175 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.320342 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.320508 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.320657 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.320805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.320961 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.320961 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.321272 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.321418 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.321570 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.321731 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.321881 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.322041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.322208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.322377 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.322542 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.322698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.322698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.322966 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.323130 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.323280 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.323429 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.323589 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.323748 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.323748 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.324106 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.324288 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.324453 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.324601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.324758 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.324925 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.325076 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.325223 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.325371 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.325537 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.325537 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.325839 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.325988 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.326202 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.326375 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.326549 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.326726 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.326874 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.327028 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.327028 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.327311 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.327447 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.327610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.327762 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.327906 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.328038 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.328459 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.328645 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.328822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.328986 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.328986 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.329215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.329310 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.329407 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.329503 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.329637 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.329803 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.329955 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.330115 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.330115 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.330416 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.330567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.330715 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.330862 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.331009 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.331278 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.331437 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.331590 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.331590 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.332336 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.332433 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.332523 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.332601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.332673 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.332745 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.332816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.332887 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.332977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.333060 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.333135 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.333207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.333332 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.333446 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.333605 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.333936 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.334041 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.334130 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.334271 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.334496 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.334631 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.334730 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.334837 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.334837 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.335042 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.335175 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.335272 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.335272 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.335414 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.335543 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.335692 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.335816 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.335906 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.336056 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.336273 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.336444 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.336634 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.336808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.336909 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.336998 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.337121 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.337218 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.337300 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.337477 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.337477 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.337636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.337712 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.337842 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.337994 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.338089 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.338238 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.338369 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.338504 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.338597 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.338723 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.338813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.338894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.339033 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.339187 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.339327 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.339431 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.339528 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.339616 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.339702 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.339985 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.340098 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.340222 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.340222 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.340393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.340529 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.340652 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.340753 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.340862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.340943 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.341022 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.341095 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.341204 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.341305 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.341393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.341478 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.341563 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.341636 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.341714 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.341845 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.341949 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.342058 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.342151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.342262 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.342356 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.342500 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.342597 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.342819 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.342918 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.343135 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.343431 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.343537 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.343769 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.343877 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.344083 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.344216 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.344425 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.344558 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.344746 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.344917 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.345081 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.345250 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.345406 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.345670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.345891 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.345999 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.346114 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.346355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.346459 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.346684 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.346828 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.347028 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.347142 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.347347 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.347635 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.347758 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.347966 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.348186 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.348345 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.348466 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.348606 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.348769 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.348987 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.349093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.349230 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.349399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.349540 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.349831 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.349970 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.350150 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.350274 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.350466 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.350556 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.350788 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.350893 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.351100 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.351194 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.351411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.351518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.351729 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.351837 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.352060 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.352249 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.352399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.352623 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.352734 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.352971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.353076 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.353286 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.353398 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.353619 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.353874 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.353978 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.354076 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.354359 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.354456 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.354677 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.354775 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.354989 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.355093 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.355375 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.355472 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.355689 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.355787 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.356019 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.356114 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.356356 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.356463 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.356670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.356768 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.356981 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.357075 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.357300 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.357406 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.357615 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.357719 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.357923 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.358025 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.358241 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.358351 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.358557 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.358662 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.358870 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.358973 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.359190 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.359311 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.359494 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.359598 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.359809 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.359906 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.360119 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.360248 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.360484 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.360582 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.360801 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.360906 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.361113 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.361287 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.361423 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.361577 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.361742 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.361895 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.362058 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.362225 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.362368 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.363337 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.363443 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.363650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.363758 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.364042 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.364225 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.364389 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.364485 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.364705 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.364820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.365023 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.365128 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.365355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.365463 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.365669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.365775 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.365987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.366088 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.366302 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.366664 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.366871 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.366977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.367182 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.367317 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.367493 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.367638 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.367817 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.367919 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.368123 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.368240 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.368468 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.368563 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.368787 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.368883 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.369116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.369214 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.369441 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.369546 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.369751 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.369849 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.370066 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.370162 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.370378 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.370528 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.370686 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.370781 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.371005 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.371111 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.371317 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.371423 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.371668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.371765 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.371979 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.372086 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.372329 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.372442 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.372667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.372772 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.373000 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.373106 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.373314 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.373420 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.373651 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.373749 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.373986 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.374092 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.374312 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.374476 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.374629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.374736 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.374952 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.375122 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.375279 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.375488 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.375595 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.375881 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.375975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.376224 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.376381 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.376537 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.376671 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.376847 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.376955 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.377157 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.377334 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.377487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.377711 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.377874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.378055 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.378189 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.378940 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.379069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.379272 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.379372 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.379587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.379704 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.379914 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.380025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.380255 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.380385 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.380563 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.380665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.380880 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.380991 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.381188 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.381306 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.381639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.381845 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.381948 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.382176 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.382285 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.382495 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.382607 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.382806 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.382912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.383127 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.383250 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.383442 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.383542 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.383761 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.383866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.384701 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.384835 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.384930 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.385018 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.385107 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.385207 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.385314 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.385418 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.385509 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.385596 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.385683 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.385933 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.386047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.386254 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.386355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.386571 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.386827 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.386924 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.387164 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.387262 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.387538 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.387647 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.387804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.387915 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.388158 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.388328 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.388479 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.388576 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.388882 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.389193 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.389193 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.389384 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.389497 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.389593 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.390188 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.390292 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.390391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.390488 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.390575 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.390739 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.390884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.391072 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.391168 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.391413 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.391563 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.391735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.391890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.392055 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.392312 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.392420 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.392643 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.392740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.393024 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.393130 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.393360 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.393453 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.393671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.393934 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.394048 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.394692 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.394804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.394900 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.395009 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.395100 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.395208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.395304 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.395399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.395543 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.395637 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.395862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.395957 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.396211 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.396309 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.396517 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.396610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.396831 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.396926 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.397145 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.397239 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.397473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.397581 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.397785 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.397882 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.398115 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.398210 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.398432 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.398533 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.398751 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.398845 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.399070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.399163 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.399561 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.399703 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.399882 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.399978 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.400234 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.400343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.400571 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.400679 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.400902 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.401001 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.401221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.401313 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.401537 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.401675 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.401860 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.401963 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.402169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.402260 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.402478 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.402573 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.402798 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.402899 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.403114 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.403216 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.403422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.403517 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.403738 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.403831 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.404051 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.404175 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.404385 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.404488 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.404709 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.404803 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.405024 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.405131 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.405444 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.405544 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.405793 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.405900 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.406094 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.406198 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.406424 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.406627 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.406832 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.406924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.407057 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.407160 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.407368 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.407463 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.407687 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.407788 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.408002 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.408096 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.408346 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.408457 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.408673 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.408768 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.408986 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.409087 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.409293 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.409386 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.409672 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.410095 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.410205 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.410302 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.410396 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.410534 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.410644 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.410851 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.410956 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.411156 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.411252 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.411486 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.411579 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.411794 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.411887 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.412104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.412265 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.412436 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.412530 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.412753 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.412857 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.413062 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.413155 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.413371 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.413463 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.413681 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.413776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.413991 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.414093 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.414308 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.414413 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.414618 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.414711 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.414929 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.415023 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.415289 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.415438 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.415535 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.415664 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.415904 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.416018 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.416250 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.416820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.416963 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.417086 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.417203 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.417479 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.417595 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.417699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.417836 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.417938 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.418028 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.418113 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.418199 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.418370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.418504 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.418602 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.418727 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.418828 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.418991 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.419094 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.419344 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.419445 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.419655 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.419755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.419971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.420228 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.420326 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.420574 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.420678 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.420904 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.421013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.421212 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.421312 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.421524 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.421617 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.421856 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.421958 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.422163 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.422264 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.422472 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.422566 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.422792 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.422925 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.423105 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.423206 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.423420 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.423514 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.423729 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.423824 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.424041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.424158 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.424380 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.424474 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.424697 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.424792 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.425012 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.425619 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.425888 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.426380 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.426488 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.426578 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.426708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.426811 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.426907 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.427004 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.427196 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.427289 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.427512 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.427604 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.427827 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.427929 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.428183 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.428295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.428514 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.428608 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.428837 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.428945 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.429160 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.429254 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.429480 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.429573 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.429806 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.430120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.430221 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.430434 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.430527 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.430754 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.430858 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.431068 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.431209 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.431387 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.431562 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.431719 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.431915 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.432037 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.432303 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.432412 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.432656 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.432794 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.433029 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.433124 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.433380 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.433488 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.433703 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.433820 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.434030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.434123 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.434356 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.434459 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.434680 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.434784 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.434995 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.435098 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.435320 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.435421 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.435634 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.435782 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.435955 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.436098 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.436307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.436411 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.436643 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.436744 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.436971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.437066 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.437283 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.437407 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.437593 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.437735 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.437900 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.438060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.438214 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.438383 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.438536 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.438702 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.438848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.439022 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.439155 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.439345 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.439447 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.439667 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.439760 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.439988 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.440089 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.440350 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.440455 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.440709 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.441569 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.442014 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.442170 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.442170 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.442342 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.442438 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.442525 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.442648 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.442828 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.442957 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.443154 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.443269 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.443535 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.443628 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.443872 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.443972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.444220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.444526 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.444668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.444834 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.444963 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.445142 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.445294 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.445449 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.445632 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.445740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.445965 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.446105 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.446297 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.446402 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.446623 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.446726 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.446958 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.447060 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.447273 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.447374 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.447587 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.447691 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.447908 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.448003 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.448251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.448347 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.448568 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.448677 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.448897 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.449035 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.449214 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.449306 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.449523 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.449618 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.449849 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.449947 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.450161 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.450253 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.450478 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.450704 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.450805 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.451064 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.451158 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.451390 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.451495 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.451717 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.451810 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.452048 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.452169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.452380 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.452474 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.452705 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.452814 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.453025 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.453117 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.453339 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.453433 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.453652 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.453760 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.453964 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.454057 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.454280 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.454387 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.454597 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.454691 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.454908 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.455003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.455222 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.455316 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.455536 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.455630 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.455845 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.455947 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.456195 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.456304 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.456619 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.456970 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.457267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.457385 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.457491 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.457591 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.457692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.457837 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.457939 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.458041 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.458148 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.458294 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.458393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.458608 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.458721 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.458923 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.459090 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.459226 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.459370 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.459539 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.459689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.459861 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.460037 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.460222 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.460395 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.460547 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.460720 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.460862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.461035 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.461182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.461343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.461549 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.461659 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=130395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 24 08:50:05.461870 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.462188 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.462518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.462839 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.463151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.463479 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.463824 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.464158 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.464474 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.464793 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.465150 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.465475 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.466041 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.466389 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.466722 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.467076 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.467416 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.467735 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.468051 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.468395 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.468732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.469055 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.469382 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.469698 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.470017 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.470337 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.470659 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.470984 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.471304 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.471622 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.472550 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.472807 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.473126 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.473445 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.473764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.474087 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.474438 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.474764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.475089 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.475413 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.475739 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.476056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.476404 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.476723 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.477041 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.477363 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.477700 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.478024 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.478337 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.478777 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.479102 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.479487 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.479845 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.480243 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.480581 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.480966 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.481340 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.481696 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.482016 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.482356 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.482759 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.483124 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.483510 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.483824 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.484150 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.484472 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.484794 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.485121 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.485438 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.485751 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.486060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.486377 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.486836 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.487004 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.487320 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.488305 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.488630 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.488947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.489254 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.489606 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.489876 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.490184 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.490496 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.490816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.491128 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.491693 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.492011 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.492350 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.492661 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.493004 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.493345 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.493672 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.494008 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.494329 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.494656 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.494977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.495307 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.495635 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.495971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.496318 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.496663 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.496988 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.497302 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.497625 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.497942 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.498258 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.498577 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.498912 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.499245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.499559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.499893 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.500240 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.500570 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.500892 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.501209 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.501545 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.501861 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.502177 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.502491 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.502807 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.503127 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.503479 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.503792 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.504116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.504459 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.504783 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.505117 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.505451 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.505771 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.506102 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.506435 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.506792 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.507162 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.507494 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.507784 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.508109 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.508450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.508762 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:50:05.509009 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=130396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 24 08:50:05.525739 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO dbcounter [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] Registered counter for database nova_api Oct 24 08:50:05.547154 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_db.sqlalchemy.engines [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130395) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:50:05.554668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writer thread running {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:50:05.574115 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO dbcounter [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] Registered counter for database nova_api Oct 24 08:50:05.594660 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_db.sqlalchemy.engines [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130396) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:50:05.605214 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writer thread running {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:50:05.612199 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.context [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),0c69bc68-c2d9-408a-9288-c4e45fffaab5(cell1) {{(pid=130395) load_cells /opt/stack/nova/nova/context.py:464}} Oct 24 08:50:05.617428 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:50:05.617905 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:50:05.619551 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:50:05.620328 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:50:05.620783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:50:05.622810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:50:05.624926 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO dbcounter [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] Registered counter for database nova_cell0 Oct 24 08:50:05.629582 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO dbcounter [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] Registered counter for database nova_cell1 Oct 24 08:50:05.639957 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_db.sqlalchemy.engines [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130395) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:50:05.641838 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_db.sqlalchemy.engines [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130395) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:50:05.654720 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writer thread running {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:50:05.656403 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writer thread running {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:50:05.667912 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.context [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),0c69bc68-c2d9-408a-9288-c4e45fffaab5(cell1) {{(pid=130396) load_cells /opt/stack/nova/nova/context.py:464}} Oct 24 08:50:05.672126 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:50:05.672634 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:50:05.674328 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:50:05.675105 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:50:05.676801 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:50:05.678952 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:50:05.681143 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO dbcounter [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] Registered counter for database nova_cell0 Oct 24 08:50:05.685336 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO dbcounter [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] Registered counter for database nova_cell1 Oct 24 08:50:05.692864 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_db.sqlalchemy.engines [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130396) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:50:05.697309 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_db.sqlalchemy.engines [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130396) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:50:05.706910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writer thread running {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:50:05.709269 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writer thread running {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:50:05.806022 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO dbcounter [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] Registered counter for database nova_cell0 Oct 24 08:50:05.813097 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_db.sqlalchemy.engines [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130395) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:50:05.818781 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writer thread running {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:50:05.865134 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO dbcounter [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] Registered counter for database nova_cell0 Oct 24 08:50:05.874914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_db.sqlalchemy.engines [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130396) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:50:05.881698 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writer thread running {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:50:06.202994 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: WARNING keystonemiddleware.auth_token [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 24 08:50:06.211696 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: WARNING keystonemiddleware.auth_token [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 24 08:50:06.232987 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: WARNING keystonemiddleware.auth_token [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 24 08:50:06.242064 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: WARNING keystonemiddleware.auth_token [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 24 08:50:06.602871 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: WARNING keystonemiddleware.auth_token [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 24 08:50:06.608890 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: WARNING keystonemiddleware.auth_token [None req-9ee15c24-7d92-409d-9af5-761c14dbcd7e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 24 08:50:06.612464 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: WARNING keystonemiddleware.auth_token [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 24 08:50:06.615062 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c51bf019c0 pid: 130395 (default app) Oct 24 08:50:06.624760 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: WARNING keystonemiddleware.auth_token [None req-25728d8c-3ddc-4c6a-a14b-faae8bf454e5 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 24 08:50:06.630795 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c51bf019c0 pid: 130396 (default app) Oct 24 08:50:49.385055 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=3 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:49.386175 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=3 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:49.387208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=3 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:49.388293 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=3 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:49.848754 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c4860cc3-2362-451e-b475-b687601fa4be tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] Calling method '.version_select at 0x7fdfeedb8af0>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:50:50.706663 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c4860cc3-2362-451e-b475-b687601fa4be tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/b26515aa-937a-406e-a7fc-6e10c110179c" status: 200 len: 351 microversion: 2.1 time: 1.334094 Oct 24 08:50:50.707346 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 1/1] 10.222.0.39 () {58 vars in 1245 bytes} [Mon Oct 24 08:50:49 2022] GET /compute/v2.1/os-volumes/b26515aa-937a-406e-a7fc-6e10c110179c => generated 351 bytes in 1337 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:50:53.036922 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-00e56a05-48cd-48e6-8473-2c3f694d4639 tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] Calling method '.version_select at 0x7fdfeee25a60>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:50:53.103789 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=3 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:53.106836 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=3 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:53.109878 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=3 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:53.117699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=3 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:54.320787 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-00e56a05-48cd-48e6-8473-2c3f694d4639 tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/3af047e9-8393-424b-9444-d6914a892f26" status: 200 len: 350 microversion: 2.1 time: 1.826868 Oct 24 08:50:54.322606 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 1/2] 10.222.0.39 () {58 vars in 1245 bytes} [Mon Oct 24 08:50:52 2022] GET /compute/v2.1/os-volumes/3af047e9-8393-424b-9444-d6914a892f26 => generated 350 bytes in 1831 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:50:55.784831 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1996623385", "imageRef": "bba331ba-faf5-4a56-81d9-6e84e9609665", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:50:56.000936 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.network.neutron [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] validate_networks() for [('51d12bec-f9d9-464c-a576-7d16d990480f', None, None, None, None, None)] {{(pid=130395) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 08:50:57.378593 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Flavor limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 08:50:57.379380 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Image limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 08:50:57.380160 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Flavor pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 08:50:57.381043 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Image pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 08:50:57.381944 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 08:50:57.443437 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:50:57.444200 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:50:57.444923 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:50:57.446089 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:50:57.446974 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:50:57.447548 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:50:57.515390 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Getting quotas for project fd4b77e78f9d4286aa92f6f90ff8b797. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:50:57.535744 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Getting quotas for user 7127613bde784330bec364f02180291e and project fd4b77e78f9d4286aa92f6f90ff8b797. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:50:57.551654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Going to run 1 instances... {{(pid=130395) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 08:50:57.555398 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bba331ba-faf5-4a56-81d9-6e84e9609665',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130395) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 08:50:57.709700 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f80be3e4-1db6-4994-9397-c03758e06af5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.267102 Oct 24 08:50:57.712155 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 2/3] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 08:50:55 2022] POST /compute/v2.1/servers => generated 383 bytes in 2276 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 08:50:57.736473 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-dd2b5877-e8a2-4e9f-a0af-85745bcd1c97 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:50:57.736473 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-dd2b5877-e8a2-4e9f-a0af-85745bcd1c97 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:50:58.014920 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-dd2b5877-e8a2-4e9f-a0af-85745bcd1c97 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:50:58.053387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-dd2b5877-e8a2-4e9f-a0af-85745bcd1c97 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1222 microversion: 2.1 time: 0.331418 Oct 24 08:50:58.058457 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 2/4] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:50:57 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1222 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:50:58.174940 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-645134853", "imageRef": "bba331ba-faf5-4a56-81d9-6e84e9609665", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:50:58.261910 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.network.neutron [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] validate_networks() for [('51d12bec-f9d9-464c-a576-7d16d990480f', None, None, None, None, None)] {{(pid=130395) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 08:50:59.582788 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Flavor limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 08:50:59.583485 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Image limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 08:50:59.584255 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Flavor pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 08:50:59.585058 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Image pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 08:50:59.585808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 08:50:59.610909 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:50:59.613445 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:50:59.615234 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:50:59.622074 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:50:59.622892 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:50:59.623707 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:50:59.669725 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Getting quotas for project fd4b77e78f9d4286aa92f6f90ff8b797. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:50:59.681633 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Getting quotas for user 7127613bde784330bec364f02180291e and project fd4b77e78f9d4286aa92f6f90ff8b797. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:50:59.709257 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Going to run 1 instances... {{(pid=130395) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 08:50:59.712306 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bba331ba-faf5-4a56-81d9-6e84e9609665',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130395) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 08:50:59.861834 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-1b216e80-8b14-4673-aa2e-887167471559 tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] Calling method '.version_select at 0x7fdfeebdd4c0>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:50:59.869725 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-dd8ecb71-026d-4c18-a5c9-966f3c3019f0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.699522 Oct 24 08:50:59.873732 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 3/5] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 08:50:58 2022] POST /compute/v2.1/servers => generated 383 bytes in 1704 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 08:50:59.882167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-bd302352-7388-49c1-993a-49bb7a0327cc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:50:59.884692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-bd302352-7388-49c1-993a-49bb7a0327cc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:00.076351 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-bd302352-7388-49c1-993a-49bb7a0327cc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:00.103379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-bd302352-7388-49c1-993a-49bb7a0327cc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1221 microversion: 2.1 time: 0.222901 Oct 24 08:51:00.129027 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 4/6] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:50:59 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1221 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:00.280614 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-1b216e80-8b14-4673-aa2e-887167471559 tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/be5d7eab-88c7-4619-a10e-123a64c59b95" status: 200 len: 350 microversion: 2.1 time: 0.419007 Oct 24 08:51:00.281868 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 3/7] 10.222.0.39 () {58 vars in 1245 bytes} [Mon Oct 24 08:50:59 2022] GET /compute/v2.1/os-volumes/be5d7eab-88c7-4619-a10e-123a64c59b95 => generated 350 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:51:00.302241 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-9e114f20-b22d-4210-89a3-aa49314edc6f tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] Calling method '.version_select at 0x7fdfee2dedc0>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:00.895478 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-9e114f20-b22d-4210-89a3-aa49314edc6f tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] 10.222.0.39 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.603060 Oct 24 08:51:00.897160 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 5/8] 10.222.0.39 () {58 vars in 1134 bytes} [Mon Oct 24 08:51:00 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 606 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:00.912919 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2b44d6d9-7812-46e1-8ab8-9b1e669403b0 tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] Calling method '.version_select at 0x7fdfeec00ca0>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:01.135469 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c4421cf6-772b-4c48-9be9-622d89cc44ec tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:01.136598 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c4421cf6-772b-4c48-9be9-622d89cc44ec tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:01.336713 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c4421cf6-772b-4c48-9be9-622d89cc44ec tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:01.353438 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c4421cf6-772b-4c48-9be9-622d89cc44ec tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1221 microversion: 2.1 time: 0.223752 Oct 24 08:51:01.357736 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 6/9] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:01 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1221 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:01.873929 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2b44d6d9-7812-46e1-8ab8-9b1e669403b0 tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] 10.222.0.39 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 693 microversion: 2.1 time: 0.965285 Oct 24 08:51:01.875033 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 4/10] 10.222.0.39 () {58 vars in 1149 bytes} [Mon Oct 24 08:51:00 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 693 bytes in 968 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:51:01.889666 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-49600c76-1e5e-4929-aaaf-589270c7051b tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] Calling method '.version_select at 0x7fdfee2f8310>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:02.241894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-49600c76-1e5e-4929-aaaf-589270c7051b tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] 10.222.0.39 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.356117 Oct 24 08:51:02.243130 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 7/11] 10.222.0.39 () {58 vars in 1134 bytes} [Mon Oct 24 08:51:01 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:02.257012 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-d8ac3592-552c-4bf1-b952-d3840778c8e1 tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] Calling method '.version_select at 0x7fdfeec00b80>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:02.397483 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-84d14c61-6180-4c32-9ea9-41987dcfece8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:02.398295 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-84d14c61-6180-4c32-9ea9-41987dcfece8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:02.423571 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84d14c61-6180-4c32-9ea9-41987dcfece8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:02.424995 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84d14c61-6180-4c32-9ea9-41987dcfece8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:02.428781 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84d14c61-6180-4c32-9ea9-41987dcfece8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:02.429700 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84d14c61-6180-4c32-9ea9-41987dcfece8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:02.430185 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84d14c61-6180-4c32-9ea9-41987dcfece8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:02.430626 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84d14c61-6180-4c32-9ea9-41987dcfece8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:02.718249 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-d8ac3592-552c-4bf1-b952-d3840778c8e1 tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] 10.222.0.39 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.464869 Oct 24 08:51:02.719316 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 5/12] 10.222.0.39 () {58 vars in 1167 bytes} [Mon Oct 24 08:51:02 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 353 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:51:02.923346 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-162468073", "imageRef": "bba331ba-faf5-4a56-81d9-6e84e9609665", "flavorRef": "452", "networks": [{"uuid": "cdaad192-fed6-4552-bf00-8c851e326b4f"}]}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:51:03.066038 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.network.neutron [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] validate_networks() for [('cdaad192-fed6-4552-bf00-8c851e326b4f', None, None, None, None, None)] {{(pid=130396) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 08:51:03.130685 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-84d14c61-6180-4c32-9ea9-41987dcfece8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:03.145181 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-84d14c61-6180-4c32-9ea9-41987dcfece8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1217 microversion: 2.1 time: 0.768938 Oct 24 08:51:03.160160 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 8/13] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:02 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1217 bytes in 771 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:03.160374 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-9a582049-f209-4df3-ac58-377ddd2296e5 tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] Calling method '.version_select at 0x7fdfee2afc10>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:03.388083 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-9a582049-f209-4df3-ac58-377ddd2296e5 tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 693 microversion: 2.1 time: 0.240544 Oct 24 08:51:03.390232 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 9/14] 10.222.0.39 () {58 vars in 1170 bytes} [Mon Oct 24 08:51:03 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 693 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:51:03.404512 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-3ecfc5fc-f286-4ff3-a0ce-33fe146aa949 tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] Calling method '.version_select at 0x7fdfee20f9d0>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:03.686692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-3ecfc5fc-f286-4ff3-a0ce-33fe146aa949 tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.286000 Oct 24 08:51:03.696010 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 10/15] 10.222.0.39 () {58 vars in 1155 bytes} [Mon Oct 24 08:51:03 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:03.702553 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-95406781-7c8d-41ea-8d6a-96ae2f09c52b tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] Calling method '.version_select at 0x7fdfee21c3a0>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:04.206317 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-95406781-7c8d-41ea-8d6a-96ae2f09c52b tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.507729 Oct 24 08:51:04.207717 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 11/16] 10.222.0.39 () {58 vars in 1188 bytes} [Mon Oct 24 08:51:03 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 353 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:51:04.214098 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-afe5081b-43fc-41c1-a80f-690414d6e51d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:04.215895 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-afe5081b-43fc-41c1-a80f-690414d6e51d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:04.248340 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-afe5081b-43fc-41c1-a80f-690414d6e51d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:04.254335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-afe5081b-43fc-41c1-a80f-690414d6e51d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:04.256002 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-afe5081b-43fc-41c1-a80f-690414d6e51d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:04.259940 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-afe5081b-43fc-41c1-a80f-690414d6e51d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:04.265913 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-afe5081b-43fc-41c1-a80f-690414d6e51d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:04.267473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-afe5081b-43fc-41c1-a80f-690414d6e51d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:04.541388 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-afe5081b-43fc-41c1-a80f-690414d6e51d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:04.550031 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-afe5081b-43fc-41c1-a80f-690414d6e51d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1279 microversion: 2.1 time: 0.340647 Oct 24 08:51:04.551800 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 12/17] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:04 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1279 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:04.559960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a6a9825f-8dac-412a-8b47-a48c52e8d2c1 tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] Calling method '.version_select at 0x7fdfee20fe50>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:04.842105 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a6a9825f-8dac-412a-8b47-a48c52e8d2c1 tempest-VolumesTestJSON-228613477 tempest-VolumesTestJSON-228613477-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.285619 Oct 24 08:51:04.842979 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 13/18] 10.222.0.39 () {58 vars in 1155 bytes} [Mon Oct 24 08:51:04 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:05.577948 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8ea49056-bb63-4f02-8aab-2cd0ec0ec188 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:05.578683 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8ea49056-bb63-4f02-8aab-2cd0ec0ec188 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:05.604645 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ea49056-bb63-4f02-8aab-2cd0ec0ec188 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:05.608374 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ea49056-bb63-4f02-8aab-2cd0ec0ec188 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:05.611928 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ea49056-bb63-4f02-8aab-2cd0ec0ec188 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:05.619801 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ea49056-bb63-4f02-8aab-2cd0ec0ec188 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:05.620610 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ea49056-bb63-4f02-8aab-2cd0ec0ec188 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:05.623207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ea49056-bb63-4f02-8aab-2cd0ec0ec188 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:06.069488 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8ea49056-bb63-4f02-8aab-2cd0ec0ec188 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:06.107155 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8ea49056-bb63-4f02-8aab-2cd0ec0ec188 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1279 microversion: 2.1 time: 0.533377 Oct 24 08:51:06.107867 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 14/19] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:05 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1279 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:06.379504 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Flavor limits 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 08:51:06.380331 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Image limits 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 08:51:06.382268 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Flavor pref 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 08:51:06.383690 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Image pref 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 08:51:06.384352 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 08:51:06.439782 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:06.448212 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:06.453672 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:06.457770 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:06.463154 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:06.475059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:06.694465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Getting quotas for project 25f1ebbfb9d042fab037dbe07a651913. Resources: {'ram', 'instances', 'cores'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:51:06.733279 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Getting quotas for user 701e4507214f4634ad09e32e21749a35 and project 25f1ebbfb9d042fab037dbe07a651913. Resources: {'ram', 'instances', 'cores'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:51:06.792765 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Going to run 1 instances... {{(pid=130396) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 08:51:06.803557 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bba331ba-faf5-4a56-81d9-6e84e9609665',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 08:51:07.118010 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-05b8dd48-f015-4bdb-861e-88ef5cf772b3 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 4.393848 Oct 24 08:51:07.118780 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 6/20] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 08:51:02 2022] POST /compute/v2.1/servers => generated 380 bytes in 4399 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 08:51:07.133407 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-44e6ff62-e226-4f06-bd8b-4d5d4726ffca tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:07.135096 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-44e6ff62-e226-4f06-bd8b-4d5d4726ffca tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:07.149052 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0ef00126-1a55-4d80-b151-ee2f73dd04d3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:07.151355 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-0ef00126-1a55-4d80-b151-ee2f73dd04d3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:07.168695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0ef00126-1a55-4d80-b151-ee2f73dd04d3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:07.169520 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0ef00126-1a55-4d80-b151-ee2f73dd04d3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:07.170796 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0ef00126-1a55-4d80-b151-ee2f73dd04d3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:07.176951 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0ef00126-1a55-4d80-b151-ee2f73dd04d3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:07.177663 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0ef00126-1a55-4d80-b151-ee2f73dd04d3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:07.178488 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0ef00126-1a55-4d80-b151-ee2f73dd04d3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:07.263194 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-44e6ff62-e226-4f06-bd8b-4d5d4726ffca tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:07.291327 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-44e6ff62-e226-4f06-bd8b-4d5d4726ffca tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1214 microversion: 2.1 time: 0.158644 Oct 24 08:51:07.293927 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 15/21] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:07 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1214 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:07.537035 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0ef00126-1a55-4d80-b151-ee2f73dd04d3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:07.595864 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-0ef00126-1a55-4d80-b151-ee2f73dd04d3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1279 microversion: 2.1 time: 0.452112 Oct 24 08:51:07.596907 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 7/22] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:07 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1279 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:08.321405 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f6ac49a0-0c8c-4cd3-8f0a-ba666c90c1d5 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:08.322466 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f6ac49a0-0c8c-4cd3-8f0a-ba666c90c1d5 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:08.616260 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f6ac49a0-0c8c-4cd3-8f0a-ba666c90c1d5 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:08.632558 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f6ac49a0-0c8c-4cd3-8f0a-ba666c90c1d5 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1214 microversion: 2.1 time: 0.315396 Oct 24 08:51:08.638685 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 8/23] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:08 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1214 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:08.657738 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f7f58059-53da-4f15-85b9-12343612ceac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:08.661344 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f7f58059-53da-4f15-85b9-12343612ceac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:08.690915 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7f58059-53da-4f15-85b9-12343612ceac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:08.691933 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7f58059-53da-4f15-85b9-12343612ceac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:08.695614 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7f58059-53da-4f15-85b9-12343612ceac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:08.697324 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7f58059-53da-4f15-85b9-12343612ceac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:08.698071 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7f58059-53da-4f15-85b9-12343612ceac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:08.698723 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7f58059-53da-4f15-85b9-12343612ceac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:08.994767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-6e2ceb07-c760-4c2e-b156-65ae17f9c247 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Action: 'create', calling method: .version_select at 0x7fdfee22f5e0>, body: {"keypair": {"name": "tempest-keypair-156035562", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCbHWu11iS14G99UpQccXOjSLqQqz9FdX3mDBRbqwQreTJK+ZNG21r+zafdQ0uwMgiYJK5ANRT4EIdJ2Z/YY0bGmvZ5WJj9uOhUTsMMbYa0RYsGABY/Xwctsu9xRj5AR8w=="}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:51:09.055256 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-6e2ceb07-c760-4c2e-b156-65ae17f9c247 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Getting quotas for project 7ff48b26c576495f8bb8c88d20d246cc. Resources: {'key_pairs'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:51:09.068242 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-6e2ceb07-c760-4c2e-b156-65ae17f9c247 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Getting quotas for user 552c23b8456a40ec8e01297aca7965d3 and project 7ff48b26c576495f8bb8c88d20d246cc. Resources: {'key_pairs'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:51:09.196396 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-6e2ceb07-c760-4c2e-b156-65ae17f9c247 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 1.086033 Oct 24 08:51:09.200004 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 16/24] 10.222.0.39 () {62 vars in 1200 bytes} [Mon Oct 24 08:51:08 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 1112 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 24 08:51:09.291222 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f7f58059-53da-4f15-85b9-12343612ceac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:09.303966 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f7f58059-53da-4f15-85b9-12343612ceac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1279 microversion: 2.1 time: 0.661760 Oct 24 08:51:09.329328 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 9/25] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:08 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1279 bytes in 689 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:09.664672 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7d793d1c-dc4c-4126-a60f-18e2d6c86139 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:09.667049 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7d793d1c-dc4c-4126-a60f-18e2d6c86139 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:09.673876 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=4 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:09.708534 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7d793d1c-dc4c-4126-a60f-18e2d6c86139 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:09.709535 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7d793d1c-dc4c-4126-a60f-18e2d6c86139 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:09.711348 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7d793d1c-dc4c-4126-a60f-18e2d6c86139 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:09.713065 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7d793d1c-dc4c-4126-a60f-18e2d6c86139 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:09.718984 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7d793d1c-dc4c-4126-a60f-18e2d6c86139 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:09.721270 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7d793d1c-dc4c-4126-a60f-18e2d6c86139 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:10.145255 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7d793d1c-dc4c-4126-a60f-18e2d6c86139 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:10.181174 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7d793d1c-dc4c-4126-a60f-18e2d6c86139 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1266 microversion: 2.1 time: 0.521972 Oct 24 08:51:10.185223 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 17/26] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:09 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1266 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:10.338715 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f28551cf-381e-49ea-8e07-5a79001369ab tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:10.339343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f28551cf-381e-49ea-8e07-5a79001369ab tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:10.346497 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f28551cf-381e-49ea-8e07-5a79001369ab tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:10.346999 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f28551cf-381e-49ea-8e07-5a79001369ab tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:10.347542 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f28551cf-381e-49ea-8e07-5a79001369ab tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:10.358373 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f28551cf-381e-49ea-8e07-5a79001369ab tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:10.361256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f28551cf-381e-49ea-8e07-5a79001369ab tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:10.363395 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f28551cf-381e-49ea-8e07-5a79001369ab tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:11.212052 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-651a4a89-728a-4275-a2a8-cae9204fa445 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:11.212882 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-651a4a89-728a-4275-a2a8-cae9204fa445 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:11.231955 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-651a4a89-728a-4275-a2a8-cae9204fa445 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:11.241548 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-651a4a89-728a-4275-a2a8-cae9204fa445 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:11.243515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-651a4a89-728a-4275-a2a8-cae9204fa445 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:11.248681 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-651a4a89-728a-4275-a2a8-cae9204fa445 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:11.254033 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-651a4a89-728a-4275-a2a8-cae9204fa445 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:11.254650 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-651a4a89-728a-4275-a2a8-cae9204fa445 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:11.484348 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f28551cf-381e-49ea-8e07-5a79001369ab tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:11.494042 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f28551cf-381e-49ea-8e07-5a79001369ab tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1279 microversion: 2.1 time: 1.160327 Oct 24 08:51:11.503934 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 10/27] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:10 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1279 bytes in 1171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:11.629509 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-651a4a89-728a-4275-a2a8-cae9204fa445 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:11.654167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-651a4a89-728a-4275-a2a8-cae9204fa445 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1272 microversion: 2.1 time: 0.443128 Oct 24 08:51:11.659610 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 18/28] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:11 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1272 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:12.200329 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5ea63ce9-85b0-40fa-87d6-287fb9d46039 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Action: 'create', calling method: .version_select at 0x7fdfee1e7820>, body: {"keypair": {"name": "tempest-keypair-1141887983", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK+k1+G4tL6pVrcmL42R5K0KKHVKZmeAJFQnUToraJ5eWSAbWYaYaDFncxi5WuPf9WFJjg410CsiO9bHPMD6vNA0sk/qTXYhL6jAjrSwIJFjEZwLPM8/Z63idIVYoRfQpw=="}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:51:12.275070 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-5ea63ce9-85b0-40fa-87d6-287fb9d46039 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Getting quotas for project b34f10ec378a4622a8f0b0546e84da7f. Resources: {'key_pairs'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:51:12.301962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-5ea63ce9-85b0-40fa-87d6-287fb9d46039 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Getting quotas for user ebdef1d69bb14837b9a0c28cbba332fc and project b34f10ec378a4622a8f0b0546e84da7f. Resources: {'key_pairs'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:51:12.398348 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-12bb2f8c-22e9-4c75-94e6-605dca2c777b req-88b2cd97-4ccd-46b7-b0c8-cc01f389848e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e7b4b4a1-6d10-4eef-94b3-395c2fc1c451", "tag": "cab5d319-dc72-4203-90f4-1a5c76d81460"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:51:12.415713 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5ea63ce9-85b0-40fa-87d6-287fb9d46039 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.320130 Oct 24 08:51:12.418057 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 11/29] 10.222.0.39 () {60 vars in 1159 bytes} [Mon Oct 24 08:51:12 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:51:12.442252 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-12bb2f8c-22e9-4c75-94e6-605dca2c777b req-88b2cd97-4ccd-46b7-b0c8-cc01f389848e service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:12.448902 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-12bb2f8c-22e9-4c75-94e6-605dca2c777b req-88b2cd97-4ccd-46b7-b0c8-cc01f389848e service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:12.450953 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-12bb2f8c-22e9-4c75-94e6-605dca2c777b req-88b2cd97-4ccd-46b7-b0c8-cc01f389848e service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:12.532222 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ddc84937-bf17-4882-9129-c20322c7e06c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:12.534341 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ddc84937-bf17-4882-9129-c20322c7e06c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:12.552025 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-12bb2f8c-22e9-4c75-94e6-605dca2c777b req-88b2cd97-4ccd-46b7-b0c8-cc01f389848e service nova] Creating event network-changed:cab5d319-dc72-4203-90f4-1a5c76d81460 for instance e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 on hv-cinder-maste Oct 24 08:51:12.552592 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-12bb2f8c-22e9-4c75-94e6-605dca2c777b req-88b2cd97-4ccd-46b7-b0c8-cc01f389848e service nova] Acquiring lock "compute-rpcapi-router" {{(pid=130395) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 24 08:51:12.557928 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-12bb2f8c-22e9-4c75-94e6-605dca2c777b req-88b2cd97-4ccd-46b7-b0c8-cc01f389848e service nova] Acquired lock "compute-rpcapi-router" {{(pid=130395) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 24 08:51:12.568861 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e3edebc9-1238-499e-8d60-52c1515dc3dd None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:12.571256 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e3edebc9-1238-499e-8d60-52c1515dc3dd None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:12.572791 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e3edebc9-1238-499e-8d60-52c1515dc3dd None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:12.575362 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e3edebc9-1238-499e-8d60-52c1515dc3dd None None] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:12.576924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e3edebc9-1238-499e-8d60-52c1515dc3dd None None] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:12.578382 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e3edebc9-1238-499e-8d60-52c1515dc3dd None None] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:12.591508 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ddc84937-bf17-4882-9129-c20322c7e06c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:12.592027 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ddc84937-bf17-4882-9129-c20322c7e06c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:12.592513 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ddc84937-bf17-4882-9129-c20322c7e06c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:12.602606 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ddc84937-bf17-4882-9129-c20322c7e06c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:12.605813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ddc84937-bf17-4882-9129-c20322c7e06c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:12.615613 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ddc84937-bf17-4882-9129-c20322c7e06c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:12.644352 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.compute.rpcapi [None req-e3edebc9-1238-499e-8d60-52c1515dc3dd None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 24 08:51:12.653149 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e3edebc9-1238-499e-8d60-52c1515dc3dd None None] Releasing lock "compute-rpcapi-router" {{(pid=130395) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 24 08:51:12.735771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e3edebc9-1238-499e-8d60-52c1515dc3dd None None] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.526695 Oct 24 08:51:12.737973 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 19/30] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:51:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:51:12.743754 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e7b839ab-c051-465e-a1b5-7d51dd1c01b6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:12.761857 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e7b839ab-c051-465e-a1b5-7d51dd1c01b6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:12.782178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7b839ab-c051-465e-a1b5-7d51dd1c01b6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:12.782587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7b839ab-c051-465e-a1b5-7d51dd1c01b6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:12.783025 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7b839ab-c051-465e-a1b5-7d51dd1c01b6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:12.783746 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7b839ab-c051-465e-a1b5-7d51dd1c01b6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:12.784222 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7b839ab-c051-465e-a1b5-7d51dd1c01b6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:12.784647 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7b839ab-c051-465e-a1b5-7d51dd1c01b6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:13.494617 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e7b839ab-c051-465e-a1b5-7d51dd1c01b6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:13.504626 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e7b839ab-c051-465e-a1b5-7d51dd1c01b6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1272 microversion: 2.1 time: 0.765122 Oct 24 08:51:13.513303 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 20/31] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:12 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1272 bytes in 775 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:14.543620 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-84ca0297-5073-412e-87ae-8d671633fa33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:14.544885 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-84ca0297-5073-412e-87ae-8d671633fa33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:14.598106 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84ca0297-5073-412e-87ae-8d671633fa33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:14.599170 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84ca0297-5073-412e-87ae-8d671633fa33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:14.599889 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84ca0297-5073-412e-87ae-8d671633fa33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:14.600899 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84ca0297-5073-412e-87ae-8d671633fa33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:14.601550 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84ca0297-5073-412e-87ae-8d671633fa33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:14.601895 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84ca0297-5073-412e-87ae-8d671633fa33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:14.786715 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ddc84937-bf17-4882-9129-c20322c7e06c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:14.803310 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ddc84937-bf17-4882-9129-c20322c7e06c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 2.276423 Oct 24 08:51:14.804672 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 12/32] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:12 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 2279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:15.419531 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-84ca0297-5073-412e-87ae-8d671633fa33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:15.430067 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-84ca0297-5073-412e-87ae-8d671633fa33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1272 microversion: 2.1 time: 0.902542 Oct 24 08:51:15.438878 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 21/33] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:14 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1272 bytes in 912 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:15.827308 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-85fe97af-04ea-4cf6-9de3-3c4378d987c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:15.827928 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-85fe97af-04ea-4cf6-9de3-3c4378d987c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:15.838112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-85fe97af-04ea-4cf6-9de3-3c4378d987c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:15.841786 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-85fe97af-04ea-4cf6-9de3-3c4378d987c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:15.842865 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-85fe97af-04ea-4cf6-9de3-3c4378d987c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:15.846206 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-85fe97af-04ea-4cf6-9de3-3c4378d987c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:15.847260 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-85fe97af-04ea-4cf6-9de3-3c4378d987c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:15.848558 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-85fe97af-04ea-4cf6-9de3-3c4378d987c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:16.088236 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9c76fe7e-f4d3-4551-8027-65c3e3c77a36 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Action: 'create', calling method: .version_select at 0x7fdfee1e7310>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-694852505", "metadata": {"Type": "work"}}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9c76fe7e-f4d3-4551-8027-65c3e3c77a36 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Request method failure captured: Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 121 Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlID_Etd2xjqZAE4_MaXKHholfBfIeyGn9nScZKO9qKd9q2QF9goc6XUSJ4KjHcwg3ZI4akW3RB9YxC4q3pxWBMX2BRsiwwKacJ2o7v7m1ih-v6SEvZsWD8MX-veGIV4Hi7WFVn6iYHK4BL1j2hktbtaxgGWZm5o3csgskWAD0LjbD4 Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 94221c3e732d4b6994f330e0320dfb05 Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 08:51:16.119932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 08:51:16.121559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/94221c3e732d4b6994f330e0320dfb05"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_94221c3e732d4b6994f330e0320dfb05", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/94221c3e732d4b6994f330e0320dfb05"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/94221c3e732d4b6994f330e0320dfb05"}]}] Oct 24 08:51:16.121559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:16.121559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 94221c3e732d4b6994f330e0320dfb05 Oct 24 08:51:16.121559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:16.121559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-VolumesNegativeTest-788051501-project Oct 24 08:51:16.121559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 08:51:16.121559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 08:51:16.121559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: b414191fa13349dc93babf241c4095d2 Oct 24 08:51:16.121559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-VolumesNegativeTest-788051501-project Oct 24 08:51:16.121559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Oct 24 08:51:16.121559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-694852505", "metadata": {"Type": "work"}}} Oct 24 08:51:16.121559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: Oct 24 08:51:16.121559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-694852505', 'metadata': {'Type': 'work'}}}} Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise error Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi '#$%' Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:51:16.122507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 24 08:51:16.123813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 24 08:51:16.125128 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 24 08:51:16.125128 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.126128 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9c76fe7e-f4d3-4551-8027-65c3e3c77a36 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:51:16.127406 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9c76fe7e-f4d3-4551-8027-65c3e3c77a36 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] 10.222.0.39 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.449657 Oct 24 08:51:16.128428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 13/34] 10.222.0.39 () {60 vars in 1156 bytes} [Mon Oct 24 08:51:15 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 452 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 24 08:51:16.144442 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-54462bd4-b8e2-4932-9e4e-72054e2cb5c5 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Action: 'create', calling method: .version_select at 0x7fdfee18b8b0>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-735512269", "metadata": {"Type": "work"}}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-54462bd4-b8e2-4932-9e4e-72054e2cb5c5 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Request method failure captured: Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 119 Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlID_Etd2xjqZAE4_MaXKHholfBfIeyGn9nScZKO9qKd9q2QF9goc6XUSJ4KjHcwg3ZI4akW3RB9YxC4q3pxWBMX2BRsiwwKacJ2o7v7m1ih-v6SEvZsWD8MX-veGIV4Hi7WFVn6iYHK4BL1j2hktbtaxgGWZm5o3csgskWAD0LjbD4 Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 94221c3e732d4b6994f330e0320dfb05 Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 08:51:16.165357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 08:51:16.166640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/94221c3e732d4b6994f330e0320dfb05"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_94221c3e732d4b6994f330e0320dfb05", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/94221c3e732d4b6994f330e0320dfb05"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/94221c3e732d4b6994f330e0320dfb05"}]}] Oct 24 08:51:16.166640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:16.166640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 94221c3e732d4b6994f330e0320dfb05 Oct 24 08:51:16.166640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:16.166640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-VolumesNegativeTest-788051501-project Oct 24 08:51:16.166640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 08:51:16.166640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 08:51:16.166640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: b414191fa13349dc93babf241c4095d2 Oct 24 08:51:16.166640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-VolumesNegativeTest-788051501-project Oct 24 08:51:16.166640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Oct 24 08:51:16.166640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-735512269", "metadata": {"Type": "work"}}} Oct 24 08:51:16.166640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: Oct 24 08:51:16.166640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-735512269', 'metadata': {'Type': 'work'}}}} Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise error Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi '0' Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:51:16.167465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 24 08:51:16.168706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 24 08:51:16.169784 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 24 08:51:16.169784 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.170169 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-54462bd4-b8e2-4932-9e4e-72054e2cb5c5 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:51:16.171155 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-54462bd4-b8e2-4932-9e4e-72054e2cb5c5 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] 10.222.0.39 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.034101 Oct 24 08:51:16.171857 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 14/35] 10.222.0.39 () {60 vars in 1156 bytes} [Mon Oct 24 08:51:16 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 35 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Oct 24 08:51:16.187394 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-970205af-8aa0-466d-9d44-74c5f056d1dd tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Action: 'create', calling method: .version_select at 0x7fdfee1e7790>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1181114415", "metadata": {"Type": "work"}}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-970205af-8aa0-466d-9d44-74c5f056d1dd tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Request method failure captured: Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 119 Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlID_Etd2xjqZAE4_MaXKHholfBfIeyGn9nScZKO9qKd9q2QF9goc6XUSJ4KjHcwg3ZI4akW3RB9YxC4q3pxWBMX2BRsiwwKacJ2o7v7m1ih-v6SEvZsWD8MX-veGIV4Hi7WFVn6iYHK4BL1j2hktbtaxgGWZm5o3csgskWAD0LjbD4 Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 94221c3e732d4b6994f330e0320dfb05 Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 08:51:16.210099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 08:51:16.211511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/94221c3e732d4b6994f330e0320dfb05"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_94221c3e732d4b6994f330e0320dfb05", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/94221c3e732d4b6994f330e0320dfb05"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/94221c3e732d4b6994f330e0320dfb05"}]}] Oct 24 08:51:16.211511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:16.211511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 94221c3e732d4b6994f330e0320dfb05 Oct 24 08:51:16.211511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:16.211511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-VolumesNegativeTest-788051501-project Oct 24 08:51:16.211511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 08:51:16.211511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 08:51:16.211511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: b414191fa13349dc93babf241c4095d2 Oct 24 08:51:16.211511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-VolumesNegativeTest-788051501-project Oct 24 08:51:16.211511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Oct 24 08:51:16.211511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1181114415", "metadata": {"Type": "work"}}} Oct 24 08:51:16.211511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: Oct 24 08:51:16.211511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1181114415', 'metadata': {'Type': 'work'}}}} Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise error Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi '' Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:51:16.212251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 24 08:51:16.213741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 24 08:51:16.215005 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 24 08:51:16.215005 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.215700 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-970205af-8aa0-466d-9d44-74c5f056d1dd tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:51:16.221548 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-970205af-8aa0-466d-9d44-74c5f056d1dd tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] 10.222.0.39 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.042918 Oct 24 08:51:16.222336 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 15/36] 10.222.0.39 () {60 vars in 1156 bytes} [Mon Oct 24 08:51:16 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 45 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 24 08:51:16.231815 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-39e29aee-bfef-4a09-970a-717f0fc1d90c tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Calling method '.version_select at 0x7fdfee161430>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:16.341317 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-39e29aee-bfef-4a09-970a-717f0fc1d90c tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] HTTP exception thrown: Volume tempest-invalid-665055505 could not be found. Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-39e29aee-bfef-4a09-970a-717f0fc1d90c tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Request method failure captured: Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-665055505 HTTP/1.1 Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlID_Etd2xjqZAE4_MaXKHholfBfIeyGn9nScZKO9qKd9q2QF9goc6XUSJ4KjHcwg3ZI4akW3RB9YxC4q3pxWBMX2BRsiwwKacJ2o7v7m1ih-v6SEvZsWD8MX-veGIV4Hi7WFVn6iYHK4BL1j2hktbtaxgGWZm5o3csgskWAD0LjbD4 Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 94221c3e732d4b6994f330e0320dfb05 Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 08:51:16.341884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 08:51:16.343090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/94221c3e732d4b6994f330e0320dfb05"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_94221c3e732d4b6994f330e0320dfb05", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/94221c3e732d4b6994f330e0320dfb05"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/94221c3e732d4b6994f330e0320dfb05"}]}] Oct 24 08:51:16.343090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:16.343090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 94221c3e732d4b6994f330e0320dfb05 Oct 24 08:51:16.343090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:16.343090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-VolumesNegativeTest-788051501-project Oct 24 08:51:16.343090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 08:51:16.343090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 08:51:16.343090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: b414191fa13349dc93babf241c4095d2 Oct 24 08:51:16.343090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-VolumesNegativeTest-788051501-project Oct 24 08:51:16.343090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: Oct 24 08:51:16.343090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Volume tempest-invalid-665055505 could not be found. Oct 24 08:51:16.343090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': 'tempest-invalid-665055505'} Oct 24 08:51:16.343090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-665055505 could not be found. (HTTP 404) (Request-ID: req-8c0474ba-255b-4f1a-85a5-72ede0316220) Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:51:16.343866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 24 08:51:16.345208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-665055505 could not be found. Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:51:16.350624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-665055505 could not be found. Oct 24 08:51:16.351911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.351911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:51:16.351911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.351911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:16.351911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:51:16.351911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:51:16.351911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:51:16.351911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:51:16.351911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-665055505 could not be found. Oct 24 08:51:16.351911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:51:16.351911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-39e29aee-bfef-4a09-970a-717f0fc1d90c tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Returning 404 to user: Volume tempest-invalid-665055505 could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:51:16.351911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-39e29aee-bfef-4a09-970a-717f0fc1d90c tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] 10.222.0.39 "DELETE /compute/v2.1/os-volumes/tempest-invalid-665055505" status: 404 len: 98 microversion: 2.1 time: 0.116579 Oct 24 08:51:16.351911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 16/37] 10.222.0.39 () {58 vars in 1215 bytes} [Mon Oct 24 08:51:16 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-665055505 => generated 98 bytes in 123 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 24 08:51:16.582803 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-06c40873-081e-4c7a-b45b-1903665344f1 req-1f10bffd-4335-4f1a-ac96-9e2701da058e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0ec5fcd8-ee4b-4143-90be-991e1da8936e", "tag": "3a7e5724-5c3f-4f69-8404-1022cccfec37"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:51:16.595286 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=2 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:16.618707 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-06c40873-081e-4c7a-b45b-1903665344f1 req-1f10bffd-4335-4f1a-ac96-9e2701da058e service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:16.621861 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-06c40873-081e-4c7a-b45b-1903665344f1 req-1f10bffd-4335-4f1a-ac96-9e2701da058e service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:16.622705 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-06c40873-081e-4c7a-b45b-1903665344f1 req-1f10bffd-4335-4f1a-ac96-9e2701da058e service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:16.665388 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-06c40873-081e-4c7a-b45b-1903665344f1 req-1f10bffd-4335-4f1a-ac96-9e2701da058e service nova] Creating event network-changed:3a7e5724-5c3f-4f69-8404-1022cccfec37 for instance 0ec5fcd8-ee4b-4143-90be-991e1da8936e on hv-cinder-maste Oct 24 08:51:16.666788 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-06c40873-081e-4c7a-b45b-1903665344f1 req-1f10bffd-4335-4f1a-ac96-9e2701da058e service nova] Acquiring lock "compute-rpcapi-router" {{(pid=130396) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 24 08:51:16.667916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-06c40873-081e-4c7a-b45b-1903665344f1 req-1f10bffd-4335-4f1a-ac96-9e2701da058e service nova] Acquired lock "compute-rpcapi-router" {{(pid=130396) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 24 08:51:16.670046 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c2b8e765-3b28-42e7-80b1-8e4616cba5f6 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:16.672425 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c2b8e765-3b28-42e7-80b1-8e4616cba5f6 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:16.674404 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c2b8e765-3b28-42e7-80b1-8e4616cba5f6 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:16.677345 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c2b8e765-3b28-42e7-80b1-8e4616cba5f6 None None] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:16.678137 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c2b8e765-3b28-42e7-80b1-8e4616cba5f6 None None] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:16.680373 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c2b8e765-3b28-42e7-80b1-8e4616cba5f6 None None] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:16.707736 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.compute.rpcapi [None req-c2b8e765-3b28-42e7-80b1-8e4616cba5f6 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 24 08:51:16.726791 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c2b8e765-3b28-42e7-80b1-8e4616cba5f6 None None] Releasing lock "compute-rpcapi-router" {{(pid=130396) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 24 08:51:16.767605 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c2b8e765-3b28-42e7-80b1-8e4616cba5f6 None None] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.415424 Oct 24 08:51:16.768854 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 17/38] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:51:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:51:16.774287 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-bcfb707c-8632-40cf-b766-0cc2e697e1d2 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] 10.222.0.39 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004029 Oct 24 08:51:16.785195 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 18/39] 10.222.0.39 () {58 vars in 1140 bytes} [Mon Oct 24 08:51:16 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 15 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 24 08:51:16.795073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a8d3338a-ea1c-4044-abd2-0cda701039b1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:16.797036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-a8d3338a-ea1c-4044-abd2-0cda701039b1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:16.818900 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a8d3338a-ea1c-4044-abd2-0cda701039b1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:16.819802 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a8d3338a-ea1c-4044-abd2-0cda701039b1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:16.820390 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a8d3338a-ea1c-4044-abd2-0cda701039b1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:16.821361 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a8d3338a-ea1c-4044-abd2-0cda701039b1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:16.822184 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a8d3338a-ea1c-4044-abd2-0cda701039b1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:16.823291 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a8d3338a-ea1c-4044-abd2-0cda701039b1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:17.066020 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-85fe97af-04ea-4cf6-9de3-3c4378d987c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:17.080976 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-85fe97af-04ea-4cf6-9de3-3c4378d987c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 1.257426 Oct 24 08:51:17.085010 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 22/40] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:15 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 1262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:17.103624 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-9c006163-51c2-4171-8a07-3a8e170260e7 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.014826 Oct 24 08:51:17.104287 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 23/41] 10.222.0.39 () {58 vars in 1137 bytes} [Mon Oct 24 08:51:17 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 19 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 24 08:51:17.139051 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f3530305-5fc4-4f81-a727-1a5e36cb1d26 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Calling method '.version_select at 0x7fdfee1ff280>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:17.230681 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-f3530305-5fc4-4f81-a727-1a5e36cb1d26 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] HTTP exception thrown: Volume 733cb482-243c-4dae-ad34-b7432f2bd4f1 could not be found. Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f3530305-5fc4-4f81-a727-1a5e36cb1d26 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Request method failure captured: Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: DELETE /compute/v2.1/os-volumes/733cb482-243c-4dae-ad34-b7432f2bd4f1 HTTP/1.1 Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlID_Etd2xjqZAE4_MaXKHholfBfIeyGn9nScZKO9qKd9q2QF9goc6XUSJ4KjHcwg3ZI4akW3RB9YxC4q3pxWBMX2BRsiwwKacJ2o7v7m1ih-v6SEvZsWD8MX-veGIV4Hi7WFVn6iYHK4BL1j2hktbtaxgGWZm5o3csgskWAD0LjbD4 Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: 94221c3e732d4b6994f330e0320dfb05 Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 08:51:17.239290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 08:51:17.240523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/94221c3e732d4b6994f330e0320dfb05"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_94221c3e732d4b6994f330e0320dfb05", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/94221c3e732d4b6994f330e0320dfb05"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/94221c3e732d4b6994f330e0320dfb05"}]}] Oct 24 08:51:17.240523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:17.240523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: 94221c3e732d4b6994f330e0320dfb05 Oct 24 08:51:17.240523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:17.240523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-VolumesNegativeTest-788051501-project Oct 24 08:51:17.240523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 08:51:17.240523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 08:51:17.240523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: b414191fa13349dc93babf241c4095d2 Oct 24 08:51:17.240523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-VolumesNegativeTest-788051501-project Oct 24 08:51:17.240523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: Oct 24 08:51:17.240523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Volume 733cb482-243c-4dae-ad34-b7432f2bd4f1 could not be found. Oct 24 08:51:17.240523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': '733cb482-243c-4dae-ad34-b7432f2bd4f1'} Oct 24 08:51:17.240523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 733cb482-243c-4dae-ad34-b7432f2bd4f1 could not be found. (HTTP 404) (Request-ID: req-29ede01c-1dca-4195-b74b-a78cbf0511b4) Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:51:17.241462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 24 08:51:17.242448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 733cb482-243c-4dae-ad34-b7432f2bd4f1 could not be found. Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 24 08:51:17.243354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:51:17.244326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 733cb482-243c-4dae-ad34-b7432f2bd4f1 could not be found. Oct 24 08:51:17.244326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:51:17.244326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:51:17.244326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:51:17.244326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:17.244326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:51:17.244326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:51:17.244326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:51:17.244326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:51:17.244326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 733cb482-243c-4dae-ad34-b7432f2bd4f1 could not be found. Oct 24 08:51:17.244326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:51:17.244326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f3530305-5fc4-4f81-a727-1a5e36cb1d26 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Returning 404 to user: Volume 733cb482-243c-4dae-ad34-b7432f2bd4f1 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:51:17.244326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f3530305-5fc4-4f81-a727-1a5e36cb1d26 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] 10.222.0.39 "DELETE /compute/v2.1/os-volumes/733cb482-243c-4dae-ad34-b7432f2bd4f1" status: 404 len: 109 microversion: 2.1 time: 0.114298 Oct 24 08:51:17.244326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 24/42] 10.222.0.39 () {58 vars in 1248 bytes} [Mon Oct 24 08:51:17 2022] DELETE /compute/v2.1/os-volumes/733cb482-243c-4dae-ad34-b7432f2bd4f1 => generated 109 bytes in 131 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 08:51:17.261759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-25c92baf-f924-4096-b902-a23ed69e0c26 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Calling method '.version_select at 0x7fdfee19c040>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:17.319920 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-25c92baf-f924-4096-b902-a23ed69e0c26 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] HTTP exception thrown: Volume 123e15b0-cbbc-47b3-a373-8b844ed9eb95 could not be found. Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-25c92baf-f924-4096-b902-a23ed69e0c26 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Request method failure captured: Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/os-volumes/123e15b0-cbbc-47b3-a373-8b844ed9eb95 HTTP/1.1 Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlID_Etd2xjqZAE4_MaXKHholfBfIeyGn9nScZKO9qKd9q2QF9goc6XUSJ4KjHcwg3ZI4akW3RB9YxC4q3pxWBMX2BRsiwwKacJ2o7v7m1ih-v6SEvZsWD8MX-veGIV4Hi7WFVn6iYHK4BL1j2hktbtaxgGWZm5o3csgskWAD0LjbD4 Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: 94221c3e732d4b6994f330e0320dfb05 Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 08:51:17.322778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 08:51:17.324192 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/94221c3e732d4b6994f330e0320dfb05"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_94221c3e732d4b6994f330e0320dfb05", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/94221c3e732d4b6994f330e0320dfb05"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/94221c3e732d4b6994f330e0320dfb05"}]}] Oct 24 08:51:17.324192 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:17.324192 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: 94221c3e732d4b6994f330e0320dfb05 Oct 24 08:51:17.324192 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-VolumesNegativeTest-788051501 Oct 24 08:51:17.324192 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-VolumesNegativeTest-788051501-project Oct 24 08:51:17.324192 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 08:51:17.324192 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 08:51:17.324192 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: b414191fa13349dc93babf241c4095d2 Oct 24 08:51:17.324192 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-VolumesNegativeTest-788051501-project Oct 24 08:51:17.324192 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: Oct 24 08:51:17.324192 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Volume 123e15b0-cbbc-47b3-a373-8b844ed9eb95 could not be found. Oct 24 08:51:17.324192 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': '123e15b0-cbbc-47b3-a373-8b844ed9eb95'} Oct 24 08:51:17.324192 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 123e15b0-cbbc-47b3-a373-8b844ed9eb95 could not be found. (HTTP 404) (Request-ID: req-3b439b7b-2d71-44d5-9c1d-f288f61874eb) Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:51:17.335462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 24 08:51:17.336601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 123e15b0-cbbc-47b3-a373-8b844ed9eb95 could not be found. Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 24 08:51:17.337778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:51:17.338874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 123e15b0-cbbc-47b3-a373-8b844ed9eb95 could not be found. Oct 24 08:51:17.338874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:51:17.338874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:51:17.338874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:51:17.338874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:51:17.338874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:51:17.338874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:51:17.338874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:51:17.338874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:51:17.338874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 123e15b0-cbbc-47b3-a373-8b844ed9eb95 could not be found. Oct 24 08:51:17.338874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:51:17.338874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-25c92baf-f924-4096-b902-a23ed69e0c26 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] Returning 404 to user: Volume 123e15b0-cbbc-47b3-a373-8b844ed9eb95 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:51:17.338874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-25c92baf-f924-4096-b902-a23ed69e0c26 tempest-VolumesNegativeTest-788051501 tempest-VolumesNegativeTest-788051501-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/123e15b0-cbbc-47b3-a373-8b844ed9eb95" status: 404 len: 109 microversion: 2.1 time: 0.075100 Oct 24 08:51:17.338874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 25/43] 10.222.0.39 () {58 vars in 1245 bytes} [Mon Oct 24 08:51:17 2022] GET /compute/v2.1/os-volumes/123e15b0-cbbc-47b3-a373-8b844ed9eb95 => generated 109 bytes in 76 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 08:51:17.517557 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-a8d3338a-ea1c-4044-abd2-0cda701039b1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:17.537237 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-a8d3338a-ea1c-4044-abd2-0cda701039b1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1272 microversion: 2.1 time: 0.750620 Oct 24 08:51:17.546667 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 19/44] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:16 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1272 bytes in 761 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:18.109226 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-67ebd359-d9c1-4462-b78d-1670711d1995 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:18.109987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-67ebd359-d9c1-4462-b78d-1670711d1995 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:18.138134 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-67ebd359-d9c1-4462-b78d-1670711d1995 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:18.139098 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-67ebd359-d9c1-4462-b78d-1670711d1995 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:18.146226 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-67ebd359-d9c1-4462-b78d-1670711d1995 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:18.147319 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-67ebd359-d9c1-4462-b78d-1670711d1995 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:18.148045 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-67ebd359-d9c1-4462-b78d-1670711d1995 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:18.148808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-67ebd359-d9c1-4462-b78d-1670711d1995 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:18.580389 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-55e1c3ed-f741-4cbc-a3ca-7589fda4cdcb tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:18.582281 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-55e1c3ed-f741-4cbc-a3ca-7589fda4cdcb tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:18.620500 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-55e1c3ed-f741-4cbc-a3ca-7589fda4cdcb tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:18.620500 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-55e1c3ed-f741-4cbc-a3ca-7589fda4cdcb tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:18.620500 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-55e1c3ed-f741-4cbc-a3ca-7589fda4cdcb tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:18.620500 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-55e1c3ed-f741-4cbc-a3ca-7589fda4cdcb tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:18.621033 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-55e1c3ed-f741-4cbc-a3ca-7589fda4cdcb tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:18.621033 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-55e1c3ed-f741-4cbc-a3ca-7589fda4cdcb tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:18.728218 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-67ebd359-d9c1-4462-b78d-1670711d1995 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:18.742392 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-67ebd359-d9c1-4462-b78d-1670711d1995 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.637747 Oct 24 08:51:18.762283 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 26/45] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:18 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 658 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:18.864121 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-55e1c3ed-f741-4cbc-a3ca-7589fda4cdcb tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:18.889982 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-55e1c3ed-f741-4cbc-a3ca-7589fda4cdcb tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1272 microversion: 2.1 time: 0.314728 Oct 24 08:51:18.903856 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 20/46] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:18 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1272 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:19.772745 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-03c39324-a77e-40e7-8bc6-78e27a1f14dd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:19.773368 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-03c39324-a77e-40e7-8bc6-78e27a1f14dd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:19.778711 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=22 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:19.792838 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-03c39324-a77e-40e7-8bc6-78e27a1f14dd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:19.804982 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-03c39324-a77e-40e7-8bc6-78e27a1f14dd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:19.808074 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-03c39324-a77e-40e7-8bc6-78e27a1f14dd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:19.815252 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-03c39324-a77e-40e7-8bc6-78e27a1f14dd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:19.818366 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-03c39324-a77e-40e7-8bc6-78e27a1f14dd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:19.819164 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-03c39324-a77e-40e7-8bc6-78e27a1f14dd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:19.925124 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-fd697bb7-33e1-474d-86d9-efce622d3585 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:19.925861 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-fd697bb7-33e1-474d-86d9-efce622d3585 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:19.939914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fd697bb7-33e1-474d-86d9-efce622d3585 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:19.940468 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fd697bb7-33e1-474d-86d9-efce622d3585 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:19.940957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fd697bb7-33e1-474d-86d9-efce622d3585 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:19.941683 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fd697bb7-33e1-474d-86d9-efce622d3585 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:19.942155 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fd697bb7-33e1-474d-86d9-efce622d3585 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:19.951343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fd697bb7-33e1-474d-86d9-efce622d3585 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:21.494201 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fd697bb7-33e1-474d-86d9-efce622d3585 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:21.513289 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-fd697bb7-33e1-474d-86d9-efce622d3585 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 1.584253 Oct 24 08:51:21.513960 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 21/47] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:19 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 1594 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:21.545781 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-03c39324-a77e-40e7-8bc6-78e27a1f14dd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:21.554022 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=67 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:21.582366 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-03c39324-a77e-40e7-8bc6-78e27a1f14dd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 1.814032 Oct 24 08:51:21.596161 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 27/48] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:19 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 1822 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:22.440933 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1678479614", "imageRef": "bba331ba-faf5-4a56-81d9-6e84e9609665", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "151b561f-53f8-4019-8ed3-b08a9e67fe57"}], "security_groups": [{"name": "tempest-securitygroup--856140750"}], "key_name": "tempest-keypair-1141887983", "user_data": "}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:51:22.449935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=14 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:22.543369 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-508af6db-e990-40d8-9eb8-ef984c62fe07 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:22.544233 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-508af6db-e990-40d8-9eb8-ef984c62fe07 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:22.546554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=72,nova_api:INSERT=7 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:22.583583 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-508af6db-e990-40d8-9eb8-ef984c62fe07 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:22.585760 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-508af6db-e990-40d8-9eb8-ef984c62fe07 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:22.587142 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-508af6db-e990-40d8-9eb8-ef984c62fe07 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:22.589721 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-508af6db-e990-40d8-9eb8-ef984c62fe07 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:22.592307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-508af6db-e990-40d8-9eb8-ef984c62fe07 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:22.594725 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-508af6db-e990-40d8-9eb8-ef984c62fe07 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:22.660768 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=2 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:22.830846 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.network.neutron [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] validate_networks() for [('151b561f-53f8-4019-8ed3-b08a9e67fe57', None, None, None, None, None)] {{(pid=130396) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 08:51:23.092674 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-508af6db-e990-40d8-9eb8-ef984c62fe07 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:23.105303 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-508af6db-e990-40d8-9eb8-ef984c62fe07 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.566878 Oct 24 08:51:23.108064 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 28/49] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:22 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:23.119407 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-da3da623-061b-43d0-9372-1380c239a9ce tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:23.120453 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-da3da623-061b-43d0-9372-1380c239a9ce tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:23.129995 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-da3da623-061b-43d0-9372-1380c239a9ce tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:23.130862 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-da3da623-061b-43d0-9372-1380c239a9ce tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:23.133368 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-da3da623-061b-43d0-9372-1380c239a9ce tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:23.135284 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-da3da623-061b-43d0-9372-1380c239a9ce tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:23.135977 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-da3da623-061b-43d0-9372-1380c239a9ce tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:23.136644 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-da3da623-061b-43d0-9372-1380c239a9ce tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:23.291846 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=51,nova_api:INSERT=4 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:23.513999 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-da3da623-061b-43d0-9372-1380c239a9ce tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:23.523373 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-da3da623-061b-43d0-9372-1380c239a9ce tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.414014 Oct 24 08:51:23.523939 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 29/50] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:23 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:23.544625 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-832584707", "imageRef": "bba331ba-faf5-4a56-81d9-6e84e9609665", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "622e8c7b-08b8-4dcd-91fb-a2d1b63f4abe"}], "security_groups": [{"name": "tempest-securitygroup--268039451"}], "key_name": "tempest-keypair-156035562", "user_data": "}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:51:23.715216 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.network.neutron [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] validate_networks() for [('622e8c7b-08b8-4dcd-91fb-a2d1b63f4abe', None, None, None, None, None)] {{(pid=130395) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 08:51:23.744182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Flavor limits 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 08:51:23.744941 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Image limits 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 08:51:23.745721 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Flavor pref 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 08:51:23.746375 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Image pref 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 08:51:23.747014 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 08:51:23.757020 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:23.757869 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:23.758514 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:23.759459 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:23.760123 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:23.760775 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:23.794449 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Getting quotas for project b34f10ec378a4622a8f0b0546e84da7f. Resources: {'ram', 'instances', 'cores'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:51:23.802686 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Getting quotas for user ebdef1d69bb14837b9a0c28cbba332fc and project b34f10ec378a4622a8f0b0546e84da7f. Resources: {'ram', 'instances', 'cores'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:51:23.826659 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Going to run 1 instances... {{(pid=130396) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 08:51:23.830022 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bba331ba-faf5-4a56-81d9-6e84e9609665',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 08:51:23.909806 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-740a1996-40b6-4b99-b138-7c22b02e7360 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.474245 Oct 24 08:51:23.910543 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 22/51] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 08:51:22 2022] POST /compute/v2.1/servers => generated 401 bytes in 1476 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 08:51:23.915393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-b4ae88b9-7845-413d-807a-da252b4a88e2 req-72a9adfa-ada6-4986-b152-b038733b7121 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fad848f1-177f-482d-a460-8a554066970c", "tag": "ac62796b-0699-4c3e-84f4-a4a808a0dd94"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:51:23.928356 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-b4ae88b9-7845-413d-807a-da252b4a88e2 req-72a9adfa-ada6-4986-b152-b038733b7121 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:23.929276 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-b4ae88b9-7845-413d-807a-da252b4a88e2 req-72a9adfa-ada6-4986-b152-b038733b7121 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:23.929727 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-b4ae88b9-7845-413d-807a-da252b4a88e2 req-72a9adfa-ada6-4986-b152-b038733b7121 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:23.943854 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-b4ae88b9-7845-413d-807a-da252b4a88e2 req-72a9adfa-ada6-4986-b152-b038733b7121 service nova] Creating event network-changed:ac62796b-0699-4c3e-84f4-a4a808a0dd94 for instance fad848f1-177f-482d-a460-8a554066970c on hv-cinder-maste Oct 24 08:51:23.950397 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-b4ae88b9-7845-413d-807a-da252b4a88e2 req-72a9adfa-ada6-4986-b152-b038733b7121 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037845 Oct 24 08:51:23.951347 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 23/52] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:51:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:51:23.958039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-1677dbd1-c001-4412-b0cd-e4db877dea12 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:23.958750 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-1677dbd1-c001-4412-b0cd-e4db877dea12 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:24.122797 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-1677dbd1-c001-4412-b0cd-e4db877dea12 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:24.134355 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-1677dbd1-c001-4412-b0cd-e4db877dea12 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1242 microversion: 2.1 time: 0.181443 Oct 24 08:51:24.134957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 24/53] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:23 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1242 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:24.149938 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a23a3fc0-8803-420c-af89-3f730d62755e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:24.151599 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-a23a3fc0-8803-420c-af89-3f730d62755e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:24.159654 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a23a3fc0-8803-420c-af89-3f730d62755e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:24.160276 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a23a3fc0-8803-420c-af89-3f730d62755e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:24.161244 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a23a3fc0-8803-420c-af89-3f730d62755e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:24.162629 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a23a3fc0-8803-420c-af89-3f730d62755e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:24.163429 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a23a3fc0-8803-420c-af89-3f730d62755e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:24.164057 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a23a3fc0-8803-420c-af89-3f730d62755e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:24.462481 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-a23a3fc0-8803-420c-af89-3f730d62755e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:24.473169 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-a23a3fc0-8803-420c-af89-3f730d62755e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.327587 Oct 24 08:51:24.473765 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 25/54] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:24 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:24.497235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Flavor limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 08:51:24.498128 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Image limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 08:51:24.498556 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Flavor pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 08:51:24.498969 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Image pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 08:51:24.499374 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 08:51:24.516623 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:24.517367 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:24.518032 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:24.519004 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:24.519789 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:24.520603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:24.540527 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Getting quotas for project 7ff48b26c576495f8bb8c88d20d246cc. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:51:24.548240 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5862c54e-59b2-4197-a44b-b90a45027458 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:24.549027 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-5862c54e-59b2-4197-a44b-b90a45027458 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:24.561600 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5862c54e-59b2-4197-a44b-b90a45027458 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:24.561600 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5862c54e-59b2-4197-a44b-b90a45027458 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:24.561600 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5862c54e-59b2-4197-a44b-b90a45027458 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:24.565731 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5862c54e-59b2-4197-a44b-b90a45027458 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:24.566249 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5862c54e-59b2-4197-a44b-b90a45027458 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:24.566696 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5862c54e-59b2-4197-a44b-b90a45027458 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:24.570705 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Getting quotas for user 552c23b8456a40ec8e01297aca7965d3 and project 7ff48b26c576495f8bb8c88d20d246cc. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:51:24.601199 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Going to run 1 instances... {{(pid=130395) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 08:51:24.605298 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bba331ba-faf5-4a56-81d9-6e84e9609665',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130395) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 08:51:24.673999 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-35caeaa3-6b75-4c7f-8d05-2491b65297fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 1.148963 Oct 24 08:51:24.682305 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 30/55] 10.222.0.39 () {62 vars in 1188 bytes} [Mon Oct 24 08:51:23 2022] POST /compute/v2.1/servers => generated 401 bytes in 1158 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 24 08:51:24.689442 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f3abe722-fe7c-4bd4-be32-e7a5d19b359f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:24.689996 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f3abe722-fe7c-4bd4-be32-e7a5d19b359f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:24.769668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f3abe722-fe7c-4bd4-be32-e7a5d19b359f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:24.779783 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5862c54e-59b2-4197-a44b-b90a45027458 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:24.780786 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f3abe722-fe7c-4bd4-be32-e7a5d19b359f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:24.783304 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f3abe722-fe7c-4bd4-be32-e7a5d19b359f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:24.784968 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f3abe722-fe7c-4bd4-be32-e7a5d19b359f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1280 microversion: 2.20 time: 0.100114 Oct 24 08:51:24.797138 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5862c54e-59b2-4197-a44b-b90a45027458 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.248780 Oct 24 08:51:24.797138 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 26/56] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:24 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:24.797955 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 31/57] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:24 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1280 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:25.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-03e8a52c-6c49-413f-9a5c-a9aba2595a27 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:25.168249 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-03e8a52c-6c49-413f-9a5c-a9aba2595a27 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:25.178600 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-03e8a52c-6c49-413f-9a5c-a9aba2595a27 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:25.179280 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-03e8a52c-6c49-413f-9a5c-a9aba2595a27 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:25.179871 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-03e8a52c-6c49-413f-9a5c-a9aba2595a27 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:25.180772 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-03e8a52c-6c49-413f-9a5c-a9aba2595a27 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:25.181233 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-03e8a52c-6c49-413f-9a5c-a9aba2595a27 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:25.181656 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-03e8a52c-6c49-413f-9a5c-a9aba2595a27 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:25.278180 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-03e8a52c-6c49-413f-9a5c-a9aba2595a27 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:25.287318 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-03e8a52c-6c49-413f-9a5c-a9aba2595a27 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1246 microversion: 2.1 time: 0.128471 Oct 24 08:51:25.288112 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 27/58] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:25 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1246 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:25.501880 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c3aefa1a-645c-47d0-9a2f-8c4929e6467b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:25.501880 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c3aefa1a-645c-47d0-9a2f-8c4929e6467b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:25.506998 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c3aefa1a-645c-47d0-9a2f-8c4929e6467b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:25.508357 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c3aefa1a-645c-47d0-9a2f-8c4929e6467b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:25.508829 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c3aefa1a-645c-47d0-9a2f-8c4929e6467b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:25.509573 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c3aefa1a-645c-47d0-9a2f-8c4929e6467b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:25.510012 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c3aefa1a-645c-47d0-9a2f-8c4929e6467b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:25.510438 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c3aefa1a-645c-47d0-9a2f-8c4929e6467b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:25.679340 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c3aefa1a-645c-47d0-9a2f-8c4929e6467b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:25.689973 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c3aefa1a-645c-47d0-9a2f-8c4929e6467b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.197062 Oct 24 08:51:25.691053 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 32/59] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:25 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:25.815207 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7799a90e-18d6-4ef6-9666-d660be8f8da0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:25.815857 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7799a90e-18d6-4ef6-9666-d660be8f8da0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:25.820965 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-26de14bd-ef46-4b42-bb38-b423dc0e5614 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:25.822792 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-26de14bd-ef46-4b42-bb38-b423dc0e5614 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:25.827594 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7799a90e-18d6-4ef6-9666-d660be8f8da0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:25.828647 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7799a90e-18d6-4ef6-9666-d660be8f8da0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:25.829368 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7799a90e-18d6-4ef6-9666-d660be8f8da0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:25.830391 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7799a90e-18d6-4ef6-9666-d660be8f8da0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:25.831612 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26de14bd-ef46-4b42-bb38-b423dc0e5614 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:25.832332 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7799a90e-18d6-4ef6-9666-d660be8f8da0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:25.832677 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26de14bd-ef46-4b42-bb38-b423dc0e5614 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:25.832782 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7799a90e-18d6-4ef6-9666-d660be8f8da0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:25.833424 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26de14bd-ef46-4b42-bb38-b423dc0e5614 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:25.834936 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26de14bd-ef46-4b42-bb38-b423dc0e5614 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:25.835573 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26de14bd-ef46-4b42-bb38-b423dc0e5614 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:25.836981 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26de14bd-ef46-4b42-bb38-b423dc0e5614 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:25.946634 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-26de14bd-ef46-4b42-bb38-b423dc0e5614 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:25.956425 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-26de14bd-ef46-4b42-bb38-b423dc0e5614 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:25.965828 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-26de14bd-ef46-4b42-bb38-b423dc0e5614 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:25.975139 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-26de14bd-ef46-4b42-bb38-b423dc0e5614 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1284 microversion: 2.20 time: 0.156564 Oct 24 08:51:25.976665 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 33/60] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:25 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1284 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:26.131880 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7799a90e-18d6-4ef6-9666-d660be8f8da0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:26.156660 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7799a90e-18d6-4ef6-9666-d660be8f8da0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.342486 Oct 24 08:51:26.159418 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 28/61] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:25 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:26.312720 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-4b9dbe7e-a584-4af4-9f54-6510b0b86a97 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:26.313338 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-4b9dbe7e-a584-4af4-9f54-6510b0b86a97 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:26.319555 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4b9dbe7e-a584-4af4-9f54-6510b0b86a97 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:26.320244 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4b9dbe7e-a584-4af4-9f54-6510b0b86a97 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:26.320832 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4b9dbe7e-a584-4af4-9f54-6510b0b86a97 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:26.321690 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4b9dbe7e-a584-4af4-9f54-6510b0b86a97 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:26.322325 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4b9dbe7e-a584-4af4-9f54-6510b0b86a97 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:26.322897 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4b9dbe7e-a584-4af4-9f54-6510b0b86a97 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:26.390577 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-4b9dbe7e-a584-4af4-9f54-6510b0b86a97 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:26.399052 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-4b9dbe7e-a584-4af4-9f54-6510b0b86a97 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1246 microversion: 2.1 time: 0.090300 Oct 24 08:51:26.399879 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 34/62] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:26 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1246 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:26.716520 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-097c6da1-e9f4-47e7-b61a-598b1d8441d4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:26.717158 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-097c6da1-e9f4-47e7-b61a-598b1d8441d4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:26.723955 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-097c6da1-e9f4-47e7-b61a-598b1d8441d4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:26.724651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-097c6da1-e9f4-47e7-b61a-598b1d8441d4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:26.725086 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-097c6da1-e9f4-47e7-b61a-598b1d8441d4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:26.725785 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-097c6da1-e9f4-47e7-b61a-598b1d8441d4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:26.726409 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-097c6da1-e9f4-47e7-b61a-598b1d8441d4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:26.726980 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-097c6da1-e9f4-47e7-b61a-598b1d8441d4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:26.889118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-097c6da1-e9f4-47e7-b61a-598b1d8441d4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:26.897993 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-097c6da1-e9f4-47e7-b61a-598b1d8441d4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.185390 Oct 24 08:51:26.898630 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 29/63] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:26 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:27.007664 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d313ab25-a12e-445f-866b-cdac16d58d0f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:27.007664 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d313ab25-a12e-445f-866b-cdac16d58d0f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:27.012378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d313ab25-a12e-445f-866b-cdac16d58d0f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:27.012890 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d313ab25-a12e-445f-866b-cdac16d58d0f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:27.013322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d313ab25-a12e-445f-866b-cdac16d58d0f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:27.013950 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d313ab25-a12e-445f-866b-cdac16d58d0f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:27.014393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d313ab25-a12e-445f-866b-cdac16d58d0f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:27.014809 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d313ab25-a12e-445f-866b-cdac16d58d0f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:27.083992 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d313ab25-a12e-445f-866b-cdac16d58d0f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:27.092085 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d313ab25-a12e-445f-866b-cdac16d58d0f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:27.093586 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d313ab25-a12e-445f-866b-cdac16d58d0f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:27.095733 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d313ab25-a12e-445f-866b-cdac16d58d0f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1276 microversion: 2.20 time: 0.094382 Oct 24 08:51:27.096659 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 35/64] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:26 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1276 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:27.189510 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-598522de-69c6-4252-9896-c279f8897f62 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:27.190123 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-598522de-69c6-4252-9896-c279f8897f62 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:27.196174 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-598522de-69c6-4252-9896-c279f8897f62 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:27.196838 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-598522de-69c6-4252-9896-c279f8897f62 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:27.197281 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-598522de-69c6-4252-9896-c279f8897f62 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:27.197972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-598522de-69c6-4252-9896-c279f8897f62 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:27.198422 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-598522de-69c6-4252-9896-c279f8897f62 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:27.198845 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-598522de-69c6-4252-9896-c279f8897f62 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:27.333087 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-598522de-69c6-4252-9896-c279f8897f62 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:27.341960 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-598522de-69c6-4252-9896-c279f8897f62 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.156295 Oct 24 08:51:27.342549 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 30/65] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:27 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:27.423090 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-527ad929-700b-4b52-8ec3-68eba8a63958 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:27.423697 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-527ad929-700b-4b52-8ec3-68eba8a63958 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:27.429539 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-527ad929-700b-4b52-8ec3-68eba8a63958 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:27.430030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-527ad929-700b-4b52-8ec3-68eba8a63958 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:27.430471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-527ad929-700b-4b52-8ec3-68eba8a63958 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:27.431166 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-527ad929-700b-4b52-8ec3-68eba8a63958 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:27.431625 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-527ad929-700b-4b52-8ec3-68eba8a63958 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:27.432048 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-527ad929-700b-4b52-8ec3-68eba8a63958 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:27.500042 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-527ad929-700b-4b52-8ec3-68eba8a63958 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:27.508795 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-527ad929-700b-4b52-8ec3-68eba8a63958 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1246 microversion: 2.1 time: 0.089945 Oct 24 08:51:27.510157 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 36/66] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:27 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1246 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:27.921081 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4d08319e-9d51-4402-8797-22cc5ceb9db2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:27.921692 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4d08319e-9d51-4402-8797-22cc5ceb9db2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:27.928015 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d08319e-9d51-4402-8797-22cc5ceb9db2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:27.928531 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d08319e-9d51-4402-8797-22cc5ceb9db2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:27.928968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d08319e-9d51-4402-8797-22cc5ceb9db2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:27.929665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d08319e-9d51-4402-8797-22cc5ceb9db2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:27.930113 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d08319e-9d51-4402-8797-22cc5ceb9db2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:27.930534 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d08319e-9d51-4402-8797-22cc5ceb9db2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:28.093443 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4d08319e-9d51-4402-8797-22cc5ceb9db2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:28.101251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4d08319e-9d51-4402-8797-22cc5ceb9db2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.183946 Oct 24 08:51:28.101822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 31/67] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:27 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:28.123878 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-02633f69-354b-4c63-9956-6438f0086851 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:28.124694 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-02633f69-354b-4c63-9956-6438f0086851 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:28.132699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02633f69-354b-4c63-9956-6438f0086851 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:28.133194 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02633f69-354b-4c63-9956-6438f0086851 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:28.133639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02633f69-354b-4c63-9956-6438f0086851 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:28.134377 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02633f69-354b-4c63-9956-6438f0086851 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:28.134833 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02633f69-354b-4c63-9956-6438f0086851 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:28.135262 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02633f69-354b-4c63-9956-6438f0086851 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:28.225533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-02633f69-354b-4c63-9956-6438f0086851 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:28.234072 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-02633f69-354b-4c63-9956-6438f0086851 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:28.236450 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-02633f69-354b-4c63-9956-6438f0086851 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:28.237617 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-02633f69-354b-4c63-9956-6438f0086851 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1276 microversion: 2.20 time: 0.118330 Oct 24 08:51:28.238179 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 37/68] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:28 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1276 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:28.366554 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9156cdd4-fa1f-4525-aec8-7dcaf78cd7c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:28.367404 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9156cdd4-fa1f-4525-aec8-7dcaf78cd7c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:28.374022 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9156cdd4-fa1f-4525-aec8-7dcaf78cd7c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:28.374509 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9156cdd4-fa1f-4525-aec8-7dcaf78cd7c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:28.374992 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9156cdd4-fa1f-4525-aec8-7dcaf78cd7c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:28.375966 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9156cdd4-fa1f-4525-aec8-7dcaf78cd7c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:28.376473 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9156cdd4-fa1f-4525-aec8-7dcaf78cd7c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:28.376900 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9156cdd4-fa1f-4525-aec8-7dcaf78cd7c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:28.518701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9156cdd4-fa1f-4525-aec8-7dcaf78cd7c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:28.522804 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=100 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:28.528283 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9156cdd4-fa1f-4525-aec8-7dcaf78cd7c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.166831 Oct 24 08:51:28.529331 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 32/69] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:28 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:28.537158 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-427e6301-f2b2-4915-b7be-ee100d4ebe91 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:28.537775 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-427e6301-f2b2-4915-b7be-ee100d4ebe91 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:28.544959 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-427e6301-f2b2-4915-b7be-ee100d4ebe91 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:28.545452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-427e6301-f2b2-4915-b7be-ee100d4ebe91 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:28.545887 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-427e6301-f2b2-4915-b7be-ee100d4ebe91 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:28.546589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-427e6301-f2b2-4915-b7be-ee100d4ebe91 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:28.547802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-427e6301-f2b2-4915-b7be-ee100d4ebe91 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:28.548367 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-427e6301-f2b2-4915-b7be-ee100d4ebe91 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:28.611760 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-427e6301-f2b2-4915-b7be-ee100d4ebe91 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:28.619689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-427e6301-f2b2-4915-b7be-ee100d4ebe91 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1238 microversion: 2.1 time: 0.091233 Oct 24 08:51:28.620383 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 38/70] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:28 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1238 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:29.125625 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e0c61377-ae74-47a4-9700-d7383d151a94 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:29.127092 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e0c61377-ae74-47a4-9700-d7383d151a94 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:29.133976 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0c61377-ae74-47a4-9700-d7383d151a94 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:29.134462 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0c61377-ae74-47a4-9700-d7383d151a94 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:29.134894 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0c61377-ae74-47a4-9700-d7383d151a94 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:29.135691 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0c61377-ae74-47a4-9700-d7383d151a94 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:29.136168 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0c61377-ae74-47a4-9700-d7383d151a94 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:29.136610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0c61377-ae74-47a4-9700-d7383d151a94 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:29.262808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-58415634-9125-4a54-bb8a-1b914d01a879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:29.263407 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-58415634-9125-4a54-bb8a-1b914d01a879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:29.270698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-58415634-9125-4a54-bb8a-1b914d01a879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:29.271474 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-58415634-9125-4a54-bb8a-1b914d01a879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:29.271996 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-58415634-9125-4a54-bb8a-1b914d01a879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:29.272789 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-58415634-9125-4a54-bb8a-1b914d01a879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:29.273311 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-58415634-9125-4a54-bb8a-1b914d01a879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:29.273798 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-58415634-9125-4a54-bb8a-1b914d01a879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:29.276964 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e0c61377-ae74-47a4-9700-d7383d151a94 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:29.285391 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e0c61377-ae74-47a4-9700-d7383d151a94 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.164292 Oct 24 08:51:29.286274 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 33/71] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:29 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:29.340953 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-58415634-9125-4a54-bb8a-1b914d01a879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:29.350251 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-58415634-9125-4a54-bb8a-1b914d01a879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:29.351707 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-58415634-9125-4a54-bb8a-1b914d01a879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:29.352840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-58415634-9125-4a54-bb8a-1b914d01a879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1276 microversion: 2.20 time: 0.093934 Oct 24 08:51:29.353394 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 39/72] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:29 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1276 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:51:29.553754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-02327e05-79a4-4019-8ba9-a07090e54ef8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:29.554800 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-02327e05-79a4-4019-8ba9-a07090e54ef8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:29.565117 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-02327e05-79a4-4019-8ba9-a07090e54ef8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:29.565897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-02327e05-79a4-4019-8ba9-a07090e54ef8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:29.566586 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-02327e05-79a4-4019-8ba9-a07090e54ef8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:29.567504 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-02327e05-79a4-4019-8ba9-a07090e54ef8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:29.568212 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-02327e05-79a4-4019-8ba9-a07090e54ef8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:29.568900 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-02327e05-79a4-4019-8ba9-a07090e54ef8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:29.647166 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b0bd6e6b-3793-4b9f-addc-c6ef8dbce374 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:29.648309 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b0bd6e6b-3793-4b9f-addc-c6ef8dbce374 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:29.656627 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b0bd6e6b-3793-4b9f-addc-c6ef8dbce374 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:29.657525 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b0bd6e6b-3793-4b9f-addc-c6ef8dbce374 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:29.658173 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b0bd6e6b-3793-4b9f-addc-c6ef8dbce374 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:29.659110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b0bd6e6b-3793-4b9f-addc-c6ef8dbce374 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:29.659775 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b0bd6e6b-3793-4b9f-addc-c6ef8dbce374 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:29.660526 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b0bd6e6b-3793-4b9f-addc-c6ef8dbce374 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:30.006803 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-b0bd6e6b-3793-4b9f-addc-c6ef8dbce374 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:30.019149 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b0bd6e6b-3793-4b9f-addc-c6ef8dbce374 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1238 microversion: 2.1 time: 0.376483 Oct 24 08:51:30.020566 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 40/73] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:29 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1238 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:30.100555 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-02327e05-79a4-4019-8ba9-a07090e54ef8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:30.108829 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-02327e05-79a4-4019-8ba9-a07090e54ef8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.560024 Oct 24 08:51:30.109404 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 34/74] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:29 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:30.308048 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-9d455706-fc5b-4a87-b719-bc3045ff8907 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:30.309588 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-9d455706-fc5b-4a87-b719-bc3045ff8907 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:30.319916 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9d455706-fc5b-4a87-b719-bc3045ff8907 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:30.322821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9d455706-fc5b-4a87-b719-bc3045ff8907 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:30.323579 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9d455706-fc5b-4a87-b719-bc3045ff8907 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:30.324600 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9d455706-fc5b-4a87-b719-bc3045ff8907 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:30.325266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9d455706-fc5b-4a87-b719-bc3045ff8907 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:30.326380 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9d455706-fc5b-4a87-b719-bc3045ff8907 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:30.379547 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-d993d57d-13fc-4b19-b63c-9fd30067a080 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:30.380359 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-d993d57d-13fc-4b19-b63c-9fd30067a080 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:30.387949 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d993d57d-13fc-4b19-b63c-9fd30067a080 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:30.389133 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d993d57d-13fc-4b19-b63c-9fd30067a080 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:30.389979 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d993d57d-13fc-4b19-b63c-9fd30067a080 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:30.399203 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d993d57d-13fc-4b19-b63c-9fd30067a080 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:30.407120 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d993d57d-13fc-4b19-b63c-9fd30067a080 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:30.408027 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d993d57d-13fc-4b19-b63c-9fd30067a080 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:30.531341 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d993d57d-13fc-4b19-b63c-9fd30067a080 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:30.541125 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d993d57d-13fc-4b19-b63c-9fd30067a080 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:30.542932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d993d57d-13fc-4b19-b63c-9fd30067a080 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:30.544061 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-d993d57d-13fc-4b19-b63c-9fd30067a080 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1340 microversion: 2.20 time: 0.168611 Oct 24 08:51:30.544660 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 35/75] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:30 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1340 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:30.622679 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-9d455706-fc5b-4a87-b719-bc3045ff8907 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:30.633322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-9d455706-fc5b-4a87-b719-bc3045ff8907 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.329051 Oct 24 08:51:30.633975 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 41/76] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:30 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:31.042638 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4310f34b-bc6d-4412-b828-b9fe06db7b44 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:31.043181 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4310f34b-bc6d-4412-b828-b9fe06db7b44 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:31.050024 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4310f34b-bc6d-4412-b828-b9fe06db7b44 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:31.050510 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4310f34b-bc6d-4412-b828-b9fe06db7b44 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:31.051138 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4310f34b-bc6d-4412-b828-b9fe06db7b44 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:31.052033 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4310f34b-bc6d-4412-b828-b9fe06db7b44 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:31.052524 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4310f34b-bc6d-4412-b828-b9fe06db7b44 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:31.053073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4310f34b-bc6d-4412-b828-b9fe06db7b44 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:31.117607 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4310f34b-bc6d-4412-b828-b9fe06db7b44 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:31.126495 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4310f34b-bc6d-4412-b828-b9fe06db7b44 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1294 microversion: 2.1 time: 0.087695 Oct 24 08:51:31.127079 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 36/77] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:31 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1294 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:31.134742 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-42f91899-2df7-469f-9042-3c22c4f12cc3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:31.135444 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-42f91899-2df7-469f-9042-3c22c4f12cc3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:31.141964 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42f91899-2df7-469f-9042-3c22c4f12cc3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:31.142526 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42f91899-2df7-469f-9042-3c22c4f12cc3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:31.143235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42f91899-2df7-469f-9042-3c22c4f12cc3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:31.144095 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42f91899-2df7-469f-9042-3c22c4f12cc3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:31.144768 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42f91899-2df7-469f-9042-3c22c4f12cc3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:31.145268 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42f91899-2df7-469f-9042-3c22c4f12cc3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:31.279334 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-42f91899-2df7-469f-9042-3c22c4f12cc3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:31.287028 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-42f91899-2df7-469f-9042-3c22c4f12cc3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.156362 Oct 24 08:51:31.287590 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 42/78] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:31 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:31.575600 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-35231f62-8540-4cc7-8ee3-97552b4604d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:31.575600 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-35231f62-8540-4cc7-8ee3-97552b4604d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:31.584532 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-35231f62-8540-4cc7-8ee3-97552b4604d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:31.585117 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-35231f62-8540-4cc7-8ee3-97552b4604d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:31.585836 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-35231f62-8540-4cc7-8ee3-97552b4604d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:31.586552 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-35231f62-8540-4cc7-8ee3-97552b4604d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:31.587023 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-35231f62-8540-4cc7-8ee3-97552b4604d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:31.587437 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-35231f62-8540-4cc7-8ee3-97552b4604d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:31.656776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-35231f62-8540-4cc7-8ee3-97552b4604d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:31.659348 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-079890ec-eeac-4ca0-ab6b-e6cbb7d75bc4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:31.660154 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-079890ec-eeac-4ca0-ab6b-e6cbb7d75bc4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:31.667262 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-35231f62-8540-4cc7-8ee3-97552b4604d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:31.667655 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-079890ec-eeac-4ca0-ab6b-e6cbb7d75bc4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:31.668541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-079890ec-eeac-4ca0-ab6b-e6cbb7d75bc4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:31.669233 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-35231f62-8540-4cc7-8ee3-97552b4604d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:31.669612 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-079890ec-eeac-4ca0-ab6b-e6cbb7d75bc4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:31.670634 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-079890ec-eeac-4ca0-ab6b-e6cbb7d75bc4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:31.671155 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-35231f62-8540-4cc7-8ee3-97552b4604d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1340 microversion: 2.20 time: 0.104914 Oct 24 08:51:31.671350 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-079890ec-eeac-4ca0-ab6b-e6cbb7d75bc4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:31.671668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-079890ec-eeac-4ca0-ab6b-e6cbb7d75bc4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:31.673820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 37/79] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:31 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1340 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:31.827151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-079890ec-eeac-4ca0-ab6b-e6cbb7d75bc4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:31.835631 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-079890ec-eeac-4ca0-ab6b-e6cbb7d75bc4 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.182032 Oct 24 08:51:31.836217 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 43/80] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:31 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:32.150361 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-3683fe9d-0a83-450a-9669-20830a5ba5ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:32.150977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-3683fe9d-0a83-450a-9669-20830a5ba5ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:32.157273 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3683fe9d-0a83-450a-9669-20830a5ba5ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:32.157769 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3683fe9d-0a83-450a-9669-20830a5ba5ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:32.158206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3683fe9d-0a83-450a-9669-20830a5ba5ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:32.158912 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3683fe9d-0a83-450a-9669-20830a5ba5ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:32.159369 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3683fe9d-0a83-450a-9669-20830a5ba5ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:32.159797 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3683fe9d-0a83-450a-9669-20830a5ba5ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:32.222134 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-3683fe9d-0a83-450a-9669-20830a5ba5ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:32.229627 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-3683fe9d-0a83-450a-9669-20830a5ba5ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1294 microversion: 2.1 time: 0.083180 Oct 24 08:51:32.230188 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 38/81] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:32 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1294 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:32.310839 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1d2048ed-d0c0-4fa0-8a4f-129616efae59 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:32.311524 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1d2048ed-d0c0-4fa0-8a4f-129616efae59 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:32.318503 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1d2048ed-d0c0-4fa0-8a4f-129616efae59 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:32.319021 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1d2048ed-d0c0-4fa0-8a4f-129616efae59 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:32.319486 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1d2048ed-d0c0-4fa0-8a4f-129616efae59 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:32.320246 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1d2048ed-d0c0-4fa0-8a4f-129616efae59 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:32.320731 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1d2048ed-d0c0-4fa0-8a4f-129616efae59 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:32.321246 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1d2048ed-d0c0-4fa0-8a4f-129616efae59 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:32.454284 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1d2048ed-d0c0-4fa0-8a4f-129616efae59 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:32.461774 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1d2048ed-d0c0-4fa0-8a4f-129616efae59 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.155203 Oct 24 08:51:32.462344 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 44/82] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:32 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:32.706493 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-57488074-b909-41fa-b31b-9e2ef98117db tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:32.707737 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-57488074-b909-41fa-b31b-9e2ef98117db tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:32.713602 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-57488074-b909-41fa-b31b-9e2ef98117db tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:32.714079 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-57488074-b909-41fa-b31b-9e2ef98117db tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:32.714506 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-57488074-b909-41fa-b31b-9e2ef98117db tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:32.715219 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-57488074-b909-41fa-b31b-9e2ef98117db tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:32.715663 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-57488074-b909-41fa-b31b-9e2ef98117db tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:32.716077 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-57488074-b909-41fa-b31b-9e2ef98117db tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:32.795762 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-57488074-b909-41fa-b31b-9e2ef98117db tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:32.806415 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-57488074-b909-41fa-b31b-9e2ef98117db tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:32.807884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-57488074-b909-41fa-b31b-9e2ef98117db tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:32.809002 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-57488074-b909-41fa-b31b-9e2ef98117db tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1340 microversion: 2.20 time: 0.107058 Oct 24 08:51:32.809547 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 39/83] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:32 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1340 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:32.861990 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-9c652bf3-1db0-4e57-8aec-b8b63486fdad tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:32.863187 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-9c652bf3-1db0-4e57-8aec-b8b63486fdad tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:32.870149 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c652bf3-1db0-4e57-8aec-b8b63486fdad tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:32.870892 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c652bf3-1db0-4e57-8aec-b8b63486fdad tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:32.871816 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c652bf3-1db0-4e57-8aec-b8b63486fdad tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:32.872658 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c652bf3-1db0-4e57-8aec-b8b63486fdad tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:32.873561 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c652bf3-1db0-4e57-8aec-b8b63486fdad tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:32.874359 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c652bf3-1db0-4e57-8aec-b8b63486fdad tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:33.029141 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-9c652bf3-1db0-4e57-8aec-b8b63486fdad tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:33.039156 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-9c652bf3-1db0-4e57-8aec-b8b63486fdad tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.183294 Oct 24 08:51:33.044545 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 45/84] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:32 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:33.251692 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-3788c0f1-68cf-492e-b11d-9c0f5922cfb1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:33.252606 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-3788c0f1-68cf-492e-b11d-9c0f5922cfb1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:33.261250 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3788c0f1-68cf-492e-b11d-9c0f5922cfb1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:33.261747 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3788c0f1-68cf-492e-b11d-9c0f5922cfb1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:33.262185 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3788c0f1-68cf-492e-b11d-9c0f5922cfb1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:33.262891 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3788c0f1-68cf-492e-b11d-9c0f5922cfb1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:33.263344 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3788c0f1-68cf-492e-b11d-9c0f5922cfb1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:33.263768 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3788c0f1-68cf-492e-b11d-9c0f5922cfb1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:33.339167 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-3788c0f1-68cf-492e-b11d-9c0f5922cfb1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:33.347591 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-3788c0f1-68cf-492e-b11d-9c0f5922cfb1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1294 microversion: 2.1 time: 0.099522 Oct 24 08:51:33.348245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 40/85] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:33 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1294 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:33.485187 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f9ed3c8a-313a-449f-8cd5-ad1521d5f418 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:33.485807 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f9ed3c8a-313a-449f-8cd5-ad1521d5f418 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:33.492121 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f9ed3c8a-313a-449f-8cd5-ad1521d5f418 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:33.492637 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f9ed3c8a-313a-449f-8cd5-ad1521d5f418 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:33.493081 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f9ed3c8a-313a-449f-8cd5-ad1521d5f418 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:33.493784 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f9ed3c8a-313a-449f-8cd5-ad1521d5f418 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:33.494241 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f9ed3c8a-313a-449f-8cd5-ad1521d5f418 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:33.494669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f9ed3c8a-313a-449f-8cd5-ad1521d5f418 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:33.498687 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=100 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:33.635392 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f9ed3c8a-313a-449f-8cd5-ad1521d5f418 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:33.643095 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f9ed3c8a-313a-449f-8cd5-ad1521d5f418 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.162357 Oct 24 08:51:33.643661 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 46/86] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:33 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:33.835268 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-dfa7dbea-18cf-4ff8-9c89-6f037e7e3916 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:33.835869 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-dfa7dbea-18cf-4ff8-9c89-6f037e7e3916 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:33.837753 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=4 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:33.843177 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dfa7dbea-18cf-4ff8-9c89-6f037e7e3916 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:33.843685 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dfa7dbea-18cf-4ff8-9c89-6f037e7e3916 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:33.844120 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dfa7dbea-18cf-4ff8-9c89-6f037e7e3916 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:33.844912 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dfa7dbea-18cf-4ff8-9c89-6f037e7e3916 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:33.845355 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dfa7dbea-18cf-4ff8-9c89-6f037e7e3916 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:33.845777 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dfa7dbea-18cf-4ff8-9c89-6f037e7e3916 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:33.918159 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-dfa7dbea-18cf-4ff8-9c89-6f037e7e3916 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:33.927880 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-dfa7dbea-18cf-4ff8-9c89-6f037e7e3916 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:33.929355 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-dfa7dbea-18cf-4ff8-9c89-6f037e7e3916 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:33.930452 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-dfa7dbea-18cf-4ff8-9c89-6f037e7e3916 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1340 microversion: 2.20 time: 0.099109 Oct 24 08:51:33.931009 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 41/87] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:33 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1340 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:34.061603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-0552595d-3b57-48ed-8358-b28a87adb0c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:34.062330 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-0552595d-3b57-48ed-8358-b28a87adb0c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:34.068458 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0552595d-3b57-48ed-8358-b28a87adb0c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:34.068936 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0552595d-3b57-48ed-8358-b28a87adb0c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:34.069396 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0552595d-3b57-48ed-8358-b28a87adb0c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:34.070087 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0552595d-3b57-48ed-8358-b28a87adb0c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:34.070533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0552595d-3b57-48ed-8358-b28a87adb0c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:34.070947 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0552595d-3b57-48ed-8358-b28a87adb0c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:34.224391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0552595d-3b57-48ed-8358-b28a87adb0c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:34.232500 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-0552595d-3b57-48ed-8358-b28a87adb0c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.174632 Oct 24 08:51:34.233075 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 47/88] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:34 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:34.373363 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e145a395-2fcc-481e-aba1-ca73d869ba09 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:34.374255 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e145a395-2fcc-481e-aba1-ca73d869ba09 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:34.379147 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=8 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:34.395304 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e145a395-2fcc-481e-aba1-ca73d869ba09 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:34.395833 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e145a395-2fcc-481e-aba1-ca73d869ba09 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:34.396294 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e145a395-2fcc-481e-aba1-ca73d869ba09 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:34.396995 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e145a395-2fcc-481e-aba1-ca73d869ba09 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:34.397467 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e145a395-2fcc-481e-aba1-ca73d869ba09 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:34.397935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e145a395-2fcc-481e-aba1-ca73d869ba09 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:34.487512 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e145a395-2fcc-481e-aba1-ca73d869ba09 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:34.496467 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e145a395-2fcc-481e-aba1-ca73d869ba09 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1294 microversion: 2.1 time: 0.129196 Oct 24 08:51:34.497305 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 42/89] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:34 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1294 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:34.663551 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=2 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:34.668216 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-519f9365-d0ac-429f-96a6-e4d53bc3810c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:34.668854 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-519f9365-d0ac-429f-96a6-e4d53bc3810c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:34.680272 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-519f9365-d0ac-429f-96a6-e4d53bc3810c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:34.681941 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-519f9365-d0ac-429f-96a6-e4d53bc3810c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:34.682396 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-519f9365-d0ac-429f-96a6-e4d53bc3810c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:34.684895 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-519f9365-d0ac-429f-96a6-e4d53bc3810c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:34.685333 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-519f9365-d0ac-429f-96a6-e4d53bc3810c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:34.685861 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-519f9365-d0ac-429f-96a6-e4d53bc3810c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:34.779642 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=8 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:34.855110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-519f9365-d0ac-429f-96a6-e4d53bc3810c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:34.863602 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-519f9365-d0ac-429f-96a6-e4d53bc3810c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.200429 Oct 24 08:51:34.863974 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 48/90] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:34 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:34.956846 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5efccb10-403c-4d36-9572-4ba482f6253d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:34.958134 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-5efccb10-403c-4d36-9572-4ba482f6253d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:34.964858 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5efccb10-403c-4d36-9572-4ba482f6253d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:34.965435 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5efccb10-403c-4d36-9572-4ba482f6253d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:34.966085 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5efccb10-403c-4d36-9572-4ba482f6253d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:34.966871 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5efccb10-403c-4d36-9572-4ba482f6253d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:34.967397 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5efccb10-403c-4d36-9572-4ba482f6253d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:34.967917 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5efccb10-403c-4d36-9572-4ba482f6253d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:35.055997 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5efccb10-403c-4d36-9572-4ba482f6253d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:35.065717 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5efccb10-403c-4d36-9572-4ba482f6253d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:35.067484 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5efccb10-403c-4d36-9572-4ba482f6253d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:35.068804 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5efccb10-403c-4d36-9572-4ba482f6253d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1340 microversion: 2.20 time: 0.117076 Oct 24 08:51:35.069362 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 43/91] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:34 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1340 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:51:35.255524 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-989315f1-1309-46b1-b177-363aba063f84 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:35.256146 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-989315f1-1309-46b1-b177-363aba063f84 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:35.262181 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-989315f1-1309-46b1-b177-363aba063f84 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:35.262666 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-989315f1-1309-46b1-b177-363aba063f84 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:35.263169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-989315f1-1309-46b1-b177-363aba063f84 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:35.263865 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-989315f1-1309-46b1-b177-363aba063f84 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:35.264351 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-989315f1-1309-46b1-b177-363aba063f84 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:35.264774 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-989315f1-1309-46b1-b177-363aba063f84 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:35.497273 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-989315f1-1309-46b1-b177-363aba063f84 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:35.515145 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-989315f1-1309-46b1-b177-363aba063f84 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.263400 Oct 24 08:51:35.517043 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 49/92] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:35 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:35.536086 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-99bc4db5-4aad-4e9f-ac02-5d970455e156 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:35.538482 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-99bc4db5-4aad-4e9f-ac02-5d970455e156 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:35.552816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99bc4db5-4aad-4e9f-ac02-5d970455e156 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:35.556415 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99bc4db5-4aad-4e9f-ac02-5d970455e156 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:35.557359 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99bc4db5-4aad-4e9f-ac02-5d970455e156 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:35.558321 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99bc4db5-4aad-4e9f-ac02-5d970455e156 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:35.559386 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99bc4db5-4aad-4e9f-ac02-5d970455e156 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:35.560126 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99bc4db5-4aad-4e9f-ac02-5d970455e156 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:35.658727 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-99bc4db5-4aad-4e9f-ac02-5d970455e156 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:35.667953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-99bc4db5-4aad-4e9f-ac02-5d970455e156 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1294 microversion: 2.1 time: 0.152879 Oct 24 08:51:35.669476 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 44/93] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:35 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1294 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:35.888413 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-4d129a87-7e6e-4d78-b696-b39134c66a94 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:35.889041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-4d129a87-7e6e-4d78-b696-b39134c66a94 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:35.896117 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d129a87-7e6e-4d78-b696-b39134c66a94 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:35.896639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d129a87-7e6e-4d78-b696-b39134c66a94 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:35.897081 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d129a87-7e6e-4d78-b696-b39134c66a94 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:35.897786 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d129a87-7e6e-4d78-b696-b39134c66a94 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:35.898242 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d129a87-7e6e-4d78-b696-b39134c66a94 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:35.898669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d129a87-7e6e-4d78-b696-b39134c66a94 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:36.071962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-4d129a87-7e6e-4d78-b696-b39134c66a94 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:36.084490 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-4d129a87-7e6e-4d78-b696-b39134c66a94 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.198697 Oct 24 08:51:36.085883 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 50/94] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:35 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:36.094971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-cdeec715-117a-4d5b-ab07-7b9159811bd1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:36.096235 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-cdeec715-117a-4d5b-ab07-7b9159811bd1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:36.103340 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cdeec715-117a-4d5b-ab07-7b9159811bd1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:36.104228 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cdeec715-117a-4d5b-ab07-7b9159811bd1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:36.105094 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cdeec715-117a-4d5b-ab07-7b9159811bd1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:36.106811 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cdeec715-117a-4d5b-ab07-7b9159811bd1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:36.107436 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cdeec715-117a-4d5b-ab07-7b9159811bd1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:36.108120 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cdeec715-117a-4d5b-ab07-7b9159811bd1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:36.185559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-cdeec715-117a-4d5b-ab07-7b9159811bd1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:36.195423 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-cdeec715-117a-4d5b-ab07-7b9159811bd1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:36.197333 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-cdeec715-117a-4d5b-ab07-7b9159811bd1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:36.198630 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-cdeec715-117a-4d5b-ab07-7b9159811bd1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1340 microversion: 2.20 time: 0.108306 Oct 24 08:51:36.199765 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 45/95] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:36 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1340 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:36.537938 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b6953629-9a5a-4829-b315-2c32a8d11aa6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:36.539066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b6953629-9a5a-4829-b315-2c32a8d11aa6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:36.545751 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6953629-9a5a-4829-b315-2c32a8d11aa6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:36.546402 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6953629-9a5a-4829-b315-2c32a8d11aa6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:36.547329 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6953629-9a5a-4829-b315-2c32a8d11aa6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:36.548212 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6953629-9a5a-4829-b315-2c32a8d11aa6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:36.548820 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6953629-9a5a-4829-b315-2c32a8d11aa6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:36.549414 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6953629-9a5a-4829-b315-2c32a8d11aa6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:36.692116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5566920f-e5a4-49e0-8762-98cdbb5f4931 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:36.693341 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-5566920f-e5a4-49e0-8762-98cdbb5f4931 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:36.702014 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5566920f-e5a4-49e0-8762-98cdbb5f4931 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:36.703329 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5566920f-e5a4-49e0-8762-98cdbb5f4931 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:36.703783 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5566920f-e5a4-49e0-8762-98cdbb5f4931 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:36.704544 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5566920f-e5a4-49e0-8762-98cdbb5f4931 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:36.705462 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5566920f-e5a4-49e0-8762-98cdbb5f4931 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:36.706347 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5566920f-e5a4-49e0-8762-98cdbb5f4931 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:36.711649 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-b6953629-9a5a-4829-b315-2c32a8d11aa6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:36.719019 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b6953629-9a5a-4829-b315-2c32a8d11aa6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.184265 Oct 24 08:51:36.720382 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 51/96] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:36 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:36.795676 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5566920f-e5a4-49e0-8762-98cdbb5f4931 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:36.803801 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5566920f-e5a4-49e0-8762-98cdbb5f4931 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1294 microversion: 2.1 time: 0.116295 Oct 24 08:51:36.805113 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 46/97] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:36 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1294 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:37.106670 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-46290ebc-dac8-424b-9928-300120421aaa tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:37.107271 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-46290ebc-dac8-424b-9928-300120421aaa tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:37.114507 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-46290ebc-dac8-424b-9928-300120421aaa tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:37.115210 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-46290ebc-dac8-424b-9928-300120421aaa tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:37.116048 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-46290ebc-dac8-424b-9928-300120421aaa tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:37.116787 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-46290ebc-dac8-424b-9928-300120421aaa tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:37.117381 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-46290ebc-dac8-424b-9928-300120421aaa tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:37.119085 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-46290ebc-dac8-424b-9928-300120421aaa tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:37.234745 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4e535c63-ccb3-4cc9-a6cb-9c8daf044e99 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:37.235765 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4e535c63-ccb3-4cc9-a6cb-9c8daf044e99 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:37.242728 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4e535c63-ccb3-4cc9-a6cb-9c8daf044e99 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:37.243481 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4e535c63-ccb3-4cc9-a6cb-9c8daf044e99 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:37.244068 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4e535c63-ccb3-4cc9-a6cb-9c8daf044e99 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:37.245231 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4e535c63-ccb3-4cc9-a6cb-9c8daf044e99 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:37.245975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4e535c63-ccb3-4cc9-a6cb-9c8daf044e99 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:37.246719 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4e535c63-ccb3-4cc9-a6cb-9c8daf044e99 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:37.275960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-46290ebc-dac8-424b-9928-300120421aaa tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:37.285203 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-46290ebc-dac8-424b-9928-300120421aaa tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.182171 Oct 24 08:51:37.297995 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 52/98] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:37 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:37.336863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4e535c63-ccb3-4cc9-a6cb-9c8daf044e99 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:37.347580 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4e535c63-ccb3-4cc9-a6cb-9c8daf044e99 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:37.349668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4e535c63-ccb3-4cc9-a6cb-9c8daf044e99 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:37.350780 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4e535c63-ccb3-4cc9-a6cb-9c8daf044e99 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1340 microversion: 2.20 time: 0.129920 Oct 24 08:51:37.351788 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 47/99] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:37 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1340 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:37.744328 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d74accd6-3d5c-4cf2-b210-949a9c5bf4c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:37.745177 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d74accd6-3d5c-4cf2-b210-949a9c5bf4c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:37.751708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d74accd6-3d5c-4cf2-b210-949a9c5bf4c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:37.752213 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d74accd6-3d5c-4cf2-b210-949a9c5bf4c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:37.752740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d74accd6-3d5c-4cf2-b210-949a9c5bf4c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:37.753525 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d74accd6-3d5c-4cf2-b210-949a9c5bf4c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:37.753973 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d74accd6-3d5c-4cf2-b210-949a9c5bf4c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:37.754396 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d74accd6-3d5c-4cf2-b210-949a9c5bf4c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:37.829911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f6f6d42e-f5c7-4b28-b518-54a28957ff18 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:37.830528 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f6f6d42e-f5c7-4b28-b518-54a28957ff18 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:37.836903 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f6f6d42e-f5c7-4b28-b518-54a28957ff18 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:37.837606 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f6f6d42e-f5c7-4b28-b518-54a28957ff18 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:37.838330 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f6f6d42e-f5c7-4b28-b518-54a28957ff18 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:37.839374 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f6f6d42e-f5c7-4b28-b518-54a28957ff18 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:37.839980 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f6f6d42e-f5c7-4b28-b518-54a28957ff18 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:37.840685 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f6f6d42e-f5c7-4b28-b518-54a28957ff18 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:37.918427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f6f6d42e-f5c7-4b28-b518-54a28957ff18 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:37.924219 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d74accd6-3d5c-4cf2-b210-949a9c5bf4c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:37.926535 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f6f6d42e-f5c7-4b28-b518-54a28957ff18 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1294 microversion: 2.1 time: 0.101059 Oct 24 08:51:37.927112 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 48/100] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:37 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1294 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:37.932786 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d74accd6-3d5c-4cf2-b210-949a9c5bf4c6 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.192525 Oct 24 08:51:37.933305 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 53/101] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:37 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:38.310577 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ca5c212d-189a-4581-89ed-bd9a5d6a16cd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:38.311196 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ca5c212d-189a-4581-89ed-bd9a5d6a16cd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:38.317644 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ca5c212d-189a-4581-89ed-bd9a5d6a16cd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:38.318132 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ca5c212d-189a-4581-89ed-bd9a5d6a16cd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:38.318569 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ca5c212d-189a-4581-89ed-bd9a5d6a16cd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:38.319259 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ca5c212d-189a-4581-89ed-bd9a5d6a16cd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:38.319713 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ca5c212d-189a-4581-89ed-bd9a5d6a16cd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:38.320167 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ca5c212d-189a-4581-89ed-bd9a5d6a16cd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:38.339107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=100 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:38.377049 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e8b86ec1-de1e-4613-9d0b-9b03fb074404 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:38.377746 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e8b86ec1-de1e-4613-9d0b-9b03fb074404 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:38.384826 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e8b86ec1-de1e-4613-9d0b-9b03fb074404 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:38.385201 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e8b86ec1-de1e-4613-9d0b-9b03fb074404 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:38.385847 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e8b86ec1-de1e-4613-9d0b-9b03fb074404 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:38.386555 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e8b86ec1-de1e-4613-9d0b-9b03fb074404 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:38.387002 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e8b86ec1-de1e-4613-9d0b-9b03fb074404 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:38.387735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e8b86ec1-de1e-4613-9d0b-9b03fb074404 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:38.469940 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e8b86ec1-de1e-4613-9d0b-9b03fb074404 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:38.479399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e8b86ec1-de1e-4613-9d0b-9b03fb074404 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:38.480891 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e8b86ec1-de1e-4613-9d0b-9b03fb074404 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:38.482007 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e8b86ec1-de1e-4613-9d0b-9b03fb074404 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1340 microversion: 2.20 time: 0.109717 Oct 24 08:51:38.482572 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 54/102] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:38 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1340 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:51:38.511186 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ca5c212d-189a-4581-89ed-bd9a5d6a16cd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:38.520538 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ca5c212d-189a-4581-89ed-bd9a5d6a16cd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.213806 Oct 24 08:51:38.521116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 49/103] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:38 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:38.955096 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f147b01f-2e13-47a7-a737-4f6640c43cdc tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:38.955636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f147b01f-2e13-47a7-a737-4f6640c43cdc tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:38.957315 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e54e81cb-612c-4f2f-bb63-7c4c0cecb8bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:38.957960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e54e81cb-612c-4f2f-bb63-7c4c0cecb8bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:38.964032 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e54e81cb-612c-4f2f-bb63-7c4c0cecb8bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:38.964753 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e54e81cb-612c-4f2f-bb63-7c4c0cecb8bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:38.965352 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e54e81cb-612c-4f2f-bb63-7c4c0cecb8bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:38.966222 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e54e81cb-612c-4f2f-bb63-7c4c0cecb8bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:38.966963 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e54e81cb-612c-4f2f-bb63-7c4c0cecb8bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:38.967699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e54e81cb-612c-4f2f-bb63-7c4c0cecb8bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:38.970157 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f147b01f-2e13-47a7-a737-4f6640c43cdc tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:38.970719 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f147b01f-2e13-47a7-a737-4f6640c43cdc tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:38.971110 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f147b01f-2e13-47a7-a737-4f6640c43cdc tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:38.972251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f147b01f-2e13-47a7-a737-4f6640c43cdc tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:38.972716 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f147b01f-2e13-47a7-a737-4f6640c43cdc tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:38.973163 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f147b01f-2e13-47a7-a737-4f6640c43cdc tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:39.088474 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e54e81cb-612c-4f2f-bb63-7c4c0cecb8bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:39.100311 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e54e81cb-612c-4f2f-bb63-7c4c0cecb8bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1294 microversion: 2.1 time: 0.152850 Oct 24 08:51:39.101482 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 55/104] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:38 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1294 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:39.174520 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f147b01f-2e13-47a7-a737-4f6640c43cdc tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:39.182592 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f147b01f-2e13-47a7-a737-4f6640c43cdc tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.231452 Oct 24 08:51:39.197144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 50/105] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:38 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:39.509013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ec829d21-015d-424c-83f9-71e8defd0a71 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:39.509840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ec829d21-015d-424c-83f9-71e8defd0a71 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:39.530821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ec829d21-015d-424c-83f9-71e8defd0a71 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:39.531313 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ec829d21-015d-424c-83f9-71e8defd0a71 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:39.531747 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ec829d21-015d-424c-83f9-71e8defd0a71 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:39.532480 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ec829d21-015d-424c-83f9-71e8defd0a71 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:39.532924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ec829d21-015d-424c-83f9-71e8defd0a71 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:39.533340 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ec829d21-015d-424c-83f9-71e8defd0a71 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:39.543943 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-961eeb7a-d534-416f-b576-1b077b55beda tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:39.544765 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-961eeb7a-d534-416f-b576-1b077b55beda tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:39.551795 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-961eeb7a-d534-416f-b576-1b077b55beda tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:39.552803 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-961eeb7a-d534-416f-b576-1b077b55beda tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:39.553356 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-961eeb7a-d534-416f-b576-1b077b55beda tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:39.553993 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-961eeb7a-d534-416f-b576-1b077b55beda tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:39.554869 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-961eeb7a-d534-416f-b576-1b077b55beda tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:39.555294 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-961eeb7a-d534-416f-b576-1b077b55beda tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:39.678919 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ec829d21-015d-424c-83f9-71e8defd0a71 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:39.703604 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ec829d21-015d-424c-83f9-71e8defd0a71 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:39.711530 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ec829d21-015d-424c-83f9-71e8defd0a71 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:39.714826 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ec829d21-015d-424c-83f9-71e8defd0a71 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1340 microversion: 2.20 time: 0.209065 Oct 24 08:51:39.732480 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 56/106] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:39 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1340 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:39.868727 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-a8524469-9306-40ce-8969-303826f97e00 req-73d68548-6e61-47a1-bf0f-3967ce62303f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e7b4b4a1-6d10-4eef-94b3-395c2fc1c451", "name": "network-vif-plugged", "status": "completed", "tag": "cab5d319-dc72-4203-90f4-1a5c76d81460"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:51:39.879745 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-a8524469-9306-40ce-8969-303826f97e00 req-73d68548-6e61-47a1-bf0f-3967ce62303f service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:39.880437 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-a8524469-9306-40ce-8969-303826f97e00 req-73d68548-6e61-47a1-bf0f-3967ce62303f service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:39.881094 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-a8524469-9306-40ce-8969-303826f97e00 req-73d68548-6e61-47a1-bf0f-3967ce62303f service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:39.894480 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-a8524469-9306-40ce-8969-303826f97e00 req-73d68548-6e61-47a1-bf0f-3967ce62303f service nova] Creating event network-vif-plugged:cab5d319-dc72-4203-90f4-1a5c76d81460 for instance e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 on hv-cinder-maste Oct 24 08:51:39.901186 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-a8524469-9306-40ce-8969-303826f97e00 req-73d68548-6e61-47a1-bf0f-3967ce62303f service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.131182 Oct 24 08:51:39.901860 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 57/107] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:51:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:51:39.910187 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-961eeb7a-d534-416f-b576-1b077b55beda tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:39.923095 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-961eeb7a-d534-416f-b576-1b077b55beda tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.382923 Oct 24 08:51:39.923711 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 51/108] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:39 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:40.126481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-6d38c39f-501a-45fc-b4a8-5ff3c97182ff tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:40.127181 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-6d38c39f-501a-45fc-b4a8-5ff3c97182ff tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:40.134479 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d38c39f-501a-45fc-b4a8-5ff3c97182ff tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:40.135251 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d38c39f-501a-45fc-b4a8-5ff3c97182ff tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:40.135845 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d38c39f-501a-45fc-b4a8-5ff3c97182ff tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:40.136569 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d38c39f-501a-45fc-b4a8-5ff3c97182ff tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:40.137019 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d38c39f-501a-45fc-b4a8-5ff3c97182ff tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:40.137437 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d38c39f-501a-45fc-b4a8-5ff3c97182ff tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:40.207128 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c758adea-1f57-4f95-8852-1cc85ad2175c tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:40.207728 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c758adea-1f57-4f95-8852-1cc85ad2175c tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:40.216182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c758adea-1f57-4f95-8852-1cc85ad2175c tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:40.216855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c758adea-1f57-4f95-8852-1cc85ad2175c tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:40.217423 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c758adea-1f57-4f95-8852-1cc85ad2175c tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:40.228924 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c758adea-1f57-4f95-8852-1cc85ad2175c tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:40.230198 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c758adea-1f57-4f95-8852-1cc85ad2175c tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:40.232341 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c758adea-1f57-4f95-8852-1cc85ad2175c tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:40.279630 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-6d38c39f-501a-45fc-b4a8-5ff3c97182ff tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:40.288389 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-6d38c39f-501a-45fc-b4a8-5ff3c97182ff tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1294 microversion: 2.1 time: 0.166433 Oct 24 08:51:40.298674 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 58/109] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:40 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1294 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:40.536873 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c758adea-1f57-4f95-8852-1cc85ad2175c tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:40.550161 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c758adea-1f57-4f95-8852-1cc85ad2175c tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.347410 Oct 24 08:51:40.551468 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 52/110] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:40 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:40.747168 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c35cea06-8527-44b6-a105-0a36829341f0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:40.747774 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c35cea06-8527-44b6-a105-0a36829341f0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:40.757139 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c35cea06-8527-44b6-a105-0a36829341f0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:40.758150 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c35cea06-8527-44b6-a105-0a36829341f0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:40.765810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c35cea06-8527-44b6-a105-0a36829341f0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:40.766809 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c35cea06-8527-44b6-a105-0a36829341f0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:40.767515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c35cea06-8527-44b6-a105-0a36829341f0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:40.767970 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c35cea06-8527-44b6-a105-0a36829341f0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:40.945429 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e44b6b48-fed1-470b-a02a-178c7ed495e0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:40.946237 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e44b6b48-fed1-470b-a02a-178c7ed495e0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:40.958812 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e44b6b48-fed1-470b-a02a-178c7ed495e0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:40.960712 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e44b6b48-fed1-470b-a02a-178c7ed495e0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:40.961404 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e44b6b48-fed1-470b-a02a-178c7ed495e0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:40.964475 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e44b6b48-fed1-470b-a02a-178c7ed495e0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:40.965527 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e44b6b48-fed1-470b-a02a-178c7ed495e0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:40.967658 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e44b6b48-fed1-470b-a02a-178c7ed495e0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:41.231827 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c35cea06-8527-44b6-a105-0a36829341f0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:41.241401 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c35cea06-8527-44b6-a105-0a36829341f0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:41.242848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c35cea06-8527-44b6-a105-0a36829341f0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:41.243988 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c35cea06-8527-44b6-a105-0a36829341f0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.500842 Oct 24 08:51:41.244558 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 59/111] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:40 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:41.324477 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-9bcc338b-8f6e-42d9-a51d-d43892d75f17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:41.325414 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-9bcc338b-8f6e-42d9-a51d-d43892d75f17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:41.332994 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9bcc338b-8f6e-42d9-a51d-d43892d75f17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:41.333597 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9bcc338b-8f6e-42d9-a51d-d43892d75f17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:41.334249 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9bcc338b-8f6e-42d9-a51d-d43892d75f17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:41.335023 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9bcc338b-8f6e-42d9-a51d-d43892d75f17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:41.335767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9bcc338b-8f6e-42d9-a51d-d43892d75f17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:41.336225 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9bcc338b-8f6e-42d9-a51d-d43892d75f17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:41.337626 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e44b6b48-fed1-470b-a02a-178c7ed495e0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:41.348335 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e44b6b48-fed1-470b-a02a-178c7ed495e0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.406776 Oct 24 08:51:41.349279 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 53/112] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:40 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:41.417362 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-9bcc338b-8f6e-42d9-a51d-d43892d75f17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:41.426148 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-9bcc338b-8f6e-42d9-a51d-d43892d75f17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1300 microversion: 2.1 time: 0.105263 Oct 24 08:51:41.426767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 60/113] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:41 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1300 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:41.574684 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e751a9e6-b921-4c46-aee5-23d92320504b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:41.575311 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e751a9e6-b921-4c46-aee5-23d92320504b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:41.582969 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e751a9e6-b921-4c46-aee5-23d92320504b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:41.583568 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e751a9e6-b921-4c46-aee5-23d92320504b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:41.584238 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e751a9e6-b921-4c46-aee5-23d92320504b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:41.585024 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e751a9e6-b921-4c46-aee5-23d92320504b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:41.585536 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e751a9e6-b921-4c46-aee5-23d92320504b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:41.585954 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e751a9e6-b921-4c46-aee5-23d92320504b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:41.723989 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-2c497067-356b-421e-9d07-d05b3a979ae4 req-82177ca9-5006-41ac-8384-6fbd894a2546 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "71e106ba-ad54-4e9c-8e42-9477c8db8cf6", "tag": "b8754c97-b769-418c-b461-714a5809d621"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:51:41.733396 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-2c497067-356b-421e-9d07-d05b3a979ae4 req-82177ca9-5006-41ac-8384-6fbd894a2546 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:41.734710 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-2c497067-356b-421e-9d07-d05b3a979ae4 req-82177ca9-5006-41ac-8384-6fbd894a2546 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:41.736078 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-2c497067-356b-421e-9d07-d05b3a979ae4 req-82177ca9-5006-41ac-8384-6fbd894a2546 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:41.746658 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e751a9e6-b921-4c46-aee5-23d92320504b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:41.747708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-2c497067-356b-421e-9d07-d05b3a979ae4 req-82177ca9-5006-41ac-8384-6fbd894a2546 service nova] Creating event network-changed:b8754c97-b769-418c-b461-714a5809d621 for instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 on hv-cinder-maste Oct 24 08:51:41.753912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-2c497067-356b-421e-9d07-d05b3a979ae4 req-82177ca9-5006-41ac-8384-6fbd894a2546 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033433 Oct 24 08:51:41.755151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 61/114] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:51:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:51:41.756759 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e751a9e6-b921-4c46-aee5-23d92320504b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.185797 Oct 24 08:51:41.757333 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 54/115] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:41 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:42.288941 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d0429889-92b7-4563-928f-8c1d0d787ec2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:42.289772 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d0429889-92b7-4563-928f-8c1d0d787ec2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:42.300381 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d0429889-92b7-4563-928f-8c1d0d787ec2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:42.301148 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d0429889-92b7-4563-928f-8c1d0d787ec2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:42.301902 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d0429889-92b7-4563-928f-8c1d0d787ec2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:42.302708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d0429889-92b7-4563-928f-8c1d0d787ec2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:42.303203 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d0429889-92b7-4563-928f-8c1d0d787ec2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:42.303711 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d0429889-92b7-4563-928f-8c1d0d787ec2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:42.374407 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4509f20d-4f9a-4225-8974-5b00e472c24a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:42.375857 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4509f20d-4f9a-4225-8974-5b00e472c24a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:42.385199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4509f20d-4f9a-4225-8974-5b00e472c24a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:42.387347 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4509f20d-4f9a-4225-8974-5b00e472c24a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:42.388665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4509f20d-4f9a-4225-8974-5b00e472c24a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:42.390055 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4509f20d-4f9a-4225-8974-5b00e472c24a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:42.391343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4509f20d-4f9a-4225-8974-5b00e472c24a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:42.391947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4509f20d-4f9a-4225-8974-5b00e472c24a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:42.501764 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d0429889-92b7-4563-928f-8c1d0d787ec2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:42.512487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d0429889-92b7-4563-928f-8c1d0d787ec2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:42.513914 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d0429889-92b7-4563-928f-8c1d0d787ec2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:42.516576 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d0429889-92b7-4563-928f-8c1d0d787ec2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.230736 Oct 24 08:51:42.517524 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 62/116] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:42 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:42.522600 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-84a4a96d-e85f-48e7-93a6-1ace1e7cc94c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:42.523195 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-84a4a96d-e85f-48e7-93a6-1ace1e7cc94c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:42.529220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84a4a96d-e85f-48e7-93a6-1ace1e7cc94c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:42.529697 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84a4a96d-e85f-48e7-93a6-1ace1e7cc94c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:42.530302 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84a4a96d-e85f-48e7-93a6-1ace1e7cc94c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:42.531028 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84a4a96d-e85f-48e7-93a6-1ace1e7cc94c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:42.531476 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84a4a96d-e85f-48e7-93a6-1ace1e7cc94c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:42.531894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-84a4a96d-e85f-48e7-93a6-1ace1e7cc94c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:42.611268 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-84a4a96d-e85f-48e7-93a6-1ace1e7cc94c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:42.614442 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4509f20d-4f9a-4225-8974-5b00e472c24a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:42.620554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-84a4a96d-e85f-48e7-93a6-1ace1e7cc94c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1300 microversion: 2.1 time: 0.101493 Oct 24 08:51:42.621132 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 63/117] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:42 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1300 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:42.634944 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4509f20d-4f9a-4225-8974-5b00e472c24a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.267988 Oct 24 08:51:42.639816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 55/118] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:42 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:42.779905 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-077004b2-f37e-4644-92f3-ed64c92591d1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:42.780722 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-077004b2-f37e-4644-92f3-ed64c92591d1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:42.786769 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-077004b2-f37e-4644-92f3-ed64c92591d1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:42.787325 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-077004b2-f37e-4644-92f3-ed64c92591d1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:42.787831 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-077004b2-f37e-4644-92f3-ed64c92591d1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:42.788615 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-077004b2-f37e-4644-92f3-ed64c92591d1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:42.789301 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-077004b2-f37e-4644-92f3-ed64c92591d1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:42.789785 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-077004b2-f37e-4644-92f3-ed64c92591d1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:42.794017 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=100 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:43.161944 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-077004b2-f37e-4644-92f3-ed64c92591d1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:43.169934 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-077004b2-f37e-4644-92f3-ed64c92591d1 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.393749 Oct 24 08:51:43.170506 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 64/119] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:42 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:43.542291 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2ebbaf4b-6855-41e8-bb3f-b28e80532831 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:43.542897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2ebbaf4b-6855-41e8-bb3f-b28e80532831 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:43.549580 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2ebbaf4b-6855-41e8-bb3f-b28e80532831 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:43.550063 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2ebbaf4b-6855-41e8-bb3f-b28e80532831 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:43.550529 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2ebbaf4b-6855-41e8-bb3f-b28e80532831 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:43.551224 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2ebbaf4b-6855-41e8-bb3f-b28e80532831 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:43.551673 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2ebbaf4b-6855-41e8-bb3f-b28e80532831 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:43.552200 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2ebbaf4b-6855-41e8-bb3f-b28e80532831 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:43.646753 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ca3c3cc6-812a-4b98-8c8f-69f6d4c6e2fc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:43.647360 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ca3c3cc6-812a-4b98-8c8f-69f6d4c6e2fc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:43.654352 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ca3c3cc6-812a-4b98-8c8f-69f6d4c6e2fc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:43.654925 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ca3c3cc6-812a-4b98-8c8f-69f6d4c6e2fc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:43.655425 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ca3c3cc6-812a-4b98-8c8f-69f6d4c6e2fc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:43.659906 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ca3c3cc6-812a-4b98-8c8f-69f6d4c6e2fc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:43.660737 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ca3c3cc6-812a-4b98-8c8f-69f6d4c6e2fc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:43.661536 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ca3c3cc6-812a-4b98-8c8f-69f6d4c6e2fc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:43.699981 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2ebbaf4b-6855-41e8-bb3f-b28e80532831 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:43.710226 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2ebbaf4b-6855-41e8-bb3f-b28e80532831 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:43.712121 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2ebbaf4b-6855-41e8-bb3f-b28e80532831 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:43.713229 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2ebbaf4b-6855-41e8-bb3f-b28e80532831 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.175049 Oct 24 08:51:43.713785 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 56/120] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:43 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:43.719314 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6a2c9791-b7e5-4583-9846-1ded0ef13117 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:43.720348 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6a2c9791-b7e5-4583-9846-1ded0ef13117 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:43.726672 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a2c9791-b7e5-4583-9846-1ded0ef13117 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:43.727397 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a2c9791-b7e5-4583-9846-1ded0ef13117 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:43.728080 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a2c9791-b7e5-4583-9846-1ded0ef13117 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:43.729188 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a2c9791-b7e5-4583-9846-1ded0ef13117 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:43.729710 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a2c9791-b7e5-4583-9846-1ded0ef13117 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:43.730378 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a2c9791-b7e5-4583-9846-1ded0ef13117 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:43.736436 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ca3c3cc6-812a-4b98-8c8f-69f6d4c6e2fc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:43.746181 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ca3c3cc6-812a-4b98-8c8f-69f6d4c6e2fc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1300 microversion: 2.1 time: 0.103027 Oct 24 08:51:43.760718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 65/121] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:43 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1300 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:43.883084 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6a2c9791-b7e5-4583-9846-1ded0ef13117 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:43.892005 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6a2c9791-b7e5-4583-9846-1ded0ef13117 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.177028 Oct 24 08:51:43.892615 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 57/122] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:43 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:44.192736 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-91ec7c4e-0ba0-4276-b5ec-9c0ff8f40d8e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:44.193346 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-91ec7c4e-0ba0-4276-b5ec-9c0ff8f40d8e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:44.199555 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91ec7c4e-0ba0-4276-b5ec-9c0ff8f40d8e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:44.200020 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91ec7c4e-0ba0-4276-b5ec-9c0ff8f40d8e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:44.200541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91ec7c4e-0ba0-4276-b5ec-9c0ff8f40d8e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:44.201236 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91ec7c4e-0ba0-4276-b5ec-9c0ff8f40d8e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:44.201710 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91ec7c4e-0ba0-4276-b5ec-9c0ff8f40d8e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:44.202174 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91ec7c4e-0ba0-4276-b5ec-9c0ff8f40d8e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:44.335459 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-91ec7c4e-0ba0-4276-b5ec-9c0ff8f40d8e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:44.343712 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-91ec7c4e-0ba0-4276-b5ec-9c0ff8f40d8e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.154262 Oct 24 08:51:44.344598 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 66/123] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:44 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:44.740434 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-22612343-10fd-48e4-a727-e25eda2abdb1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:44.741673 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-22612343-10fd-48e4-a727-e25eda2abdb1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:44.749116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-22612343-10fd-48e4-a727-e25eda2abdb1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:44.749613 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-22612343-10fd-48e4-a727-e25eda2abdb1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:44.750147 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-22612343-10fd-48e4-a727-e25eda2abdb1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:44.753602 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-22612343-10fd-48e4-a727-e25eda2abdb1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:44.754068 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-22612343-10fd-48e4-a727-e25eda2abdb1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:44.754502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-22612343-10fd-48e4-a727-e25eda2abdb1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:44.771341 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-9a8b51ab-e0f6-4be4-be50-f09b68df7b8c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:44.772521 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-9a8b51ab-e0f6-4be4-be50-f09b68df7b8c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:44.778528 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9a8b51ab-e0f6-4be4-be50-f09b68df7b8c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:44.779012 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9a8b51ab-e0f6-4be4-be50-f09b68df7b8c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:44.779479 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9a8b51ab-e0f6-4be4-be50-f09b68df7b8c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:44.780194 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9a8b51ab-e0f6-4be4-be50-f09b68df7b8c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:44.781556 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9a8b51ab-e0f6-4be4-be50-f09b68df7b8c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:44.781978 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9a8b51ab-e0f6-4be4-be50-f09b68df7b8c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:44.884679 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-9a8b51ab-e0f6-4be4-be50-f09b68df7b8c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:44.896323 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-9a8b51ab-e0f6-4be4-be50-f09b68df7b8c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1300 microversion: 2.1 time: 0.129662 Oct 24 08:51:44.897072 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 67/124] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:44 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1300 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:44.916481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ade1d550-7b94-4e8b-b0e2-051f5c4cba6c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:44.917091 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ade1d550-7b94-4e8b-b0e2-051f5c4cba6c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:44.923666 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ade1d550-7b94-4e8b-b0e2-051f5c4cba6c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:44.924180 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ade1d550-7b94-4e8b-b0e2-051f5c4cba6c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:44.924799 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ade1d550-7b94-4e8b-b0e2-051f5c4cba6c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:44.925587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ade1d550-7b94-4e8b-b0e2-051f5c4cba6c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:44.925968 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ade1d550-7b94-4e8b-b0e2-051f5c4cba6c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:44.926383 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ade1d550-7b94-4e8b-b0e2-051f5c4cba6c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:44.960443 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-22612343-10fd-48e4-a727-e25eda2abdb1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:44.970768 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-22612343-10fd-48e4-a727-e25eda2abdb1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:44.972245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-22612343-10fd-48e4-a727-e25eda2abdb1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:44.973599 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-22612343-10fd-48e4-a727-e25eda2abdb1 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.238209 Oct 24 08:51:44.979788 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 58/125] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:44 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:45.069253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ade1d550-7b94-4e8b-b0e2-051f5c4cba6c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:45.076807 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ade1d550-7b94-4e8b-b0e2-051f5c4cba6c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.164361 Oct 24 08:51:45.077472 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 68/126] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:44 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:45.366537 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-506bab34-e657-4a58-9e12-844de51d7a32 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:45.367247 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-506bab34-e657-4a58-9e12-844de51d7a32 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:45.373473 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-506bab34-e657-4a58-9e12-844de51d7a32 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:45.373967 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-506bab34-e657-4a58-9e12-844de51d7a32 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:45.374464 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-506bab34-e657-4a58-9e12-844de51d7a32 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:45.375685 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-506bab34-e657-4a58-9e12-844de51d7a32 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:45.376143 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-506bab34-e657-4a58-9e12-844de51d7a32 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:45.376576 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-506bab34-e657-4a58-9e12-844de51d7a32 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:45.518875 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-506bab34-e657-4a58-9e12-844de51d7a32 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:45.526770 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-506bab34-e657-4a58-9e12-844de51d7a32 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.164278 Oct 24 08:51:45.527343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 59/127] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:45 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:45.920591 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1611a27d-bb7a-4ac7-8a54-ec641e7fd7fd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:45.928686 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1611a27d-bb7a-4ac7-8a54-ec641e7fd7fd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:45.945545 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:45.947521 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1611a27d-bb7a-4ac7-8a54-ec641e7fd7fd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:45.950505 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1611a27d-bb7a-4ac7-8a54-ec641e7fd7fd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:45.950973 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1611a27d-bb7a-4ac7-8a54-ec641e7fd7fd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:45.955723 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1611a27d-bb7a-4ac7-8a54-ec641e7fd7fd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:45.958394 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1611a27d-bb7a-4ac7-8a54-ec641e7fd7fd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:45.958858 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1611a27d-bb7a-4ac7-8a54-ec641e7fd7fd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:46.005705 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2026430d-da23-4538-a49b-8fdf357b2bfd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:46.006496 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2026430d-da23-4538-a49b-8fdf357b2bfd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:46.027525 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2026430d-da23-4538-a49b-8fdf357b2bfd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:46.031467 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2026430d-da23-4538-a49b-8fdf357b2bfd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:46.036507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2026430d-da23-4538-a49b-8fdf357b2bfd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:46.038608 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2026430d-da23-4538-a49b-8fdf357b2bfd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:46.039170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2026430d-da23-4538-a49b-8fdf357b2bfd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:46.040846 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2026430d-da23-4538-a49b-8fdf357b2bfd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:46.061544 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1611a27d-bb7a-4ac7-8a54-ec641e7fd7fd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:46.070454 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1611a27d-bb7a-4ac7-8a54-ec641e7fd7fd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1300 microversion: 2.1 time: 0.154114 Oct 24 08:51:46.071267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 69/128] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:45 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1300 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:46.100893 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-000bdbe4-6e8d-4d2e-9222-e05de316997d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:46.101881 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-000bdbe4-6e8d-4d2e-9222-e05de316997d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:46.108535 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-000bdbe4-6e8d-4d2e-9222-e05de316997d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:46.109540 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-000bdbe4-6e8d-4d2e-9222-e05de316997d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:46.110156 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-000bdbe4-6e8d-4d2e-9222-e05de316997d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:46.111023 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-000bdbe4-6e8d-4d2e-9222-e05de316997d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:46.111471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-000bdbe4-6e8d-4d2e-9222-e05de316997d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:46.112109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-000bdbe4-6e8d-4d2e-9222-e05de316997d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:46.288698 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2026430d-da23-4538-a49b-8fdf357b2bfd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:46.304574 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2026430d-da23-4538-a49b-8fdf357b2bfd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:46.305927 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2026430d-da23-4538-a49b-8fdf357b2bfd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:46.307729 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2026430d-da23-4538-a49b-8fdf357b2bfd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.309635 Oct 24 08:51:46.308333 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 60/129] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:45 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:46.317757 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-000bdbe4-6e8d-4d2e-9222-e05de316997d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:46.326373 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-000bdbe4-6e8d-4d2e-9222-e05de316997d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.230165 Oct 24 08:51:46.326933 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 70/130] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:46 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:46.551384 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-fa69362f-15ab-4544-9dd7-de61e8a73624 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:46.552091 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-fa69362f-15ab-4544-9dd7-de61e8a73624 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:46.558113 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fa69362f-15ab-4544-9dd7-de61e8a73624 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:46.558538 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fa69362f-15ab-4544-9dd7-de61e8a73624 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:46.558978 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fa69362f-15ab-4544-9dd7-de61e8a73624 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:46.559654 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fa69362f-15ab-4544-9dd7-de61e8a73624 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:46.560109 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fa69362f-15ab-4544-9dd7-de61e8a73624 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:46.560568 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fa69362f-15ab-4544-9dd7-de61e8a73624 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:46.701029 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fa69362f-15ab-4544-9dd7-de61e8a73624 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:46.710073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-fa69362f-15ab-4544-9dd7-de61e8a73624 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.162541 Oct 24 08:51:46.710639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 61/131] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:46 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:47.095585 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-44a8922a-bcfd-4394-8651-a4cba084e6f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:47.096093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-44a8922a-bcfd-4394-8651-a4cba084e6f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:47.102093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-44a8922a-bcfd-4394-8651-a4cba084e6f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:47.102565 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-44a8922a-bcfd-4394-8651-a4cba084e6f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:47.103098 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-44a8922a-bcfd-4394-8651-a4cba084e6f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:47.103811 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-44a8922a-bcfd-4394-8651-a4cba084e6f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:47.104289 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-44a8922a-bcfd-4394-8651-a4cba084e6f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:47.104711 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-44a8922a-bcfd-4394-8651-a4cba084e6f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:47.169793 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-44a8922a-bcfd-4394-8651-a4cba084e6f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:47.178447 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-44a8922a-bcfd-4394-8651-a4cba084e6f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1300 microversion: 2.1 time: 0.087821 Oct 24 08:51:47.179145 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 71/132] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:47 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1300 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:47.335589 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-76816228-deca-4c94-899b-806fded02372 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:47.336604 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-76816228-deca-4c94-899b-806fded02372 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:47.347898 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76816228-deca-4c94-899b-806fded02372 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:47.348766 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76816228-deca-4c94-899b-806fded02372 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:47.349295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76816228-deca-4c94-899b-806fded02372 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:47.350118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76816228-deca-4c94-899b-806fded02372 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:47.350584 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76816228-deca-4c94-899b-806fded02372 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:47.350849 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-6c82cf54-61a0-4810-bcae-fc75ddf9dbb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:47.351022 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76816228-deca-4c94-899b-806fded02372 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:47.351464 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-6c82cf54-61a0-4810-bcae-fc75ddf9dbb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:47.357745 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6c82cf54-61a0-4810-bcae-fc75ddf9dbb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:47.358463 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6c82cf54-61a0-4810-bcae-fc75ddf9dbb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:47.359786 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6c82cf54-61a0-4810-bcae-fc75ddf9dbb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:47.360709 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6c82cf54-61a0-4810-bcae-fc75ddf9dbb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:47.361309 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6c82cf54-61a0-4810-bcae-fc75ddf9dbb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:47.362133 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6c82cf54-61a0-4810-bcae-fc75ddf9dbb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:47.585374 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-76816228-deca-4c94-899b-806fded02372 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:47.598617 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-76816228-deca-4c94-899b-806fded02372 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:47.600421 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-76816228-deca-4c94-899b-806fded02372 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:47.601923 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-76816228-deca-4c94-899b-806fded02372 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.271057 Oct 24 08:51:47.602706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 62/133] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:47 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:47.604940 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-6c82cf54-61a0-4810-bcae-fc75ddf9dbb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:47.613799 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-6c82cf54-61a0-4810-bcae-fc75ddf9dbb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.266656 Oct 24 08:51:47.615856 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 72/134] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:47 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:47.624986 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-fb418e49-a287-4e73-838d-60aa2a4d8491 req-8b338b5a-decb-4beb-8785-4b8e33ff1594 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0ec5fcd8-ee4b-4143-90be-991e1da8936e", "name": "network-vif-plugged", "status": "completed", "tag": "3a7e5724-5c3f-4f69-8404-1022cccfec37"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:51:47.635956 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-fb418e49-a287-4e73-838d-60aa2a4d8491 req-8b338b5a-decb-4beb-8785-4b8e33ff1594 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:47.636376 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-fb418e49-a287-4e73-838d-60aa2a4d8491 req-8b338b5a-decb-4beb-8785-4b8e33ff1594 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:47.636792 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-fb418e49-a287-4e73-838d-60aa2a4d8491 req-8b338b5a-decb-4beb-8785-4b8e33ff1594 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:47.649114 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-fb418e49-a287-4e73-838d-60aa2a4d8491 req-8b338b5a-decb-4beb-8785-4b8e33ff1594 service nova] Creating event network-vif-plugged:3a7e5724-5c3f-4f69-8404-1022cccfec37 for instance 0ec5fcd8-ee4b-4143-90be-991e1da8936e on hv-cinder-maste Oct 24 08:51:47.654138 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-fb418e49-a287-4e73-838d-60aa2a4d8491 req-8b338b5a-decb-4beb-8785-4b8e33ff1594 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036711 Oct 24 08:51:47.654709 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 63/135] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:51:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:51:47.732390 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1232db64-031f-4228-b173-288af51b8da2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:47.732992 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1232db64-031f-4228-b173-288af51b8da2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:47.738920 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1232db64-031f-4228-b173-288af51b8da2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:47.739395 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1232db64-031f-4228-b173-288af51b8da2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:47.739842 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1232db64-031f-4228-b173-288af51b8da2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:47.740541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1232db64-031f-4228-b173-288af51b8da2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:47.741255 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1232db64-031f-4228-b173-288af51b8da2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:47.741672 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1232db64-031f-4228-b173-288af51b8da2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:47.873338 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1232db64-031f-4228-b173-288af51b8da2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:47.881636 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1232db64-031f-4228-b173-288af51b8da2 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.153187 Oct 24 08:51:47.882197 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 73/136] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:47 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:48.203536 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-158022bf-a591-413c-b2b8-1b804ccdfeb9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:48.204382 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-158022bf-a591-413c-b2b8-1b804ccdfeb9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:48.206438 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:48.213985 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-158022bf-a591-413c-b2b8-1b804ccdfeb9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:48.214474 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-158022bf-a591-413c-b2b8-1b804ccdfeb9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:48.214903 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-158022bf-a591-413c-b2b8-1b804ccdfeb9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:48.215608 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-158022bf-a591-413c-b2b8-1b804ccdfeb9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:48.216054 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-158022bf-a591-413c-b2b8-1b804ccdfeb9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:48.216497 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-158022bf-a591-413c-b2b8-1b804ccdfeb9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:48.285648 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-158022bf-a591-413c-b2b8-1b804ccdfeb9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:48.293282 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-158022bf-a591-413c-b2b8-1b804ccdfeb9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1300 microversion: 2.1 time: 0.094778 Oct 24 08:51:48.293843 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 64/137] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:48 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1300 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:48.632448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-4dea0e61-50fc-42ea-a6b0-af15a8c4df91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:48.633061 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-4dea0e61-50fc-42ea-a6b0-af15a8c4df91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:48.638104 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9c5f7563-0fd8-47d3-878e-21ba2f4dfdbf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:48.638715 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9c5f7563-0fd8-47d3-878e-21ba2f4dfdbf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:48.639052 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4dea0e61-50fc-42ea-a6b0-af15a8c4df91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:48.639537 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4dea0e61-50fc-42ea-a6b0-af15a8c4df91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:48.640062 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4dea0e61-50fc-42ea-a6b0-af15a8c4df91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:48.640790 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4dea0e61-50fc-42ea-a6b0-af15a8c4df91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:48.641246 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4dea0e61-50fc-42ea-a6b0-af15a8c4df91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:48.641666 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4dea0e61-50fc-42ea-a6b0-af15a8c4df91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:48.645780 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c5f7563-0fd8-47d3-878e-21ba2f4dfdbf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:48.646065 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c5f7563-0fd8-47d3-878e-21ba2f4dfdbf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:48.646665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c5f7563-0fd8-47d3-878e-21ba2f4dfdbf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:48.647539 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c5f7563-0fd8-47d3-878e-21ba2f4dfdbf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:48.647996 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c5f7563-0fd8-47d3-878e-21ba2f4dfdbf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:48.648446 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c5f7563-0fd8-47d3-878e-21ba2f4dfdbf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:48.842544 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-4dea0e61-50fc-42ea-a6b0-af15a8c4df91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:48.845906 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9c5f7563-0fd8-47d3-878e-21ba2f4dfdbf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:48.851866 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-4dea0e61-50fc-42ea-a6b0-af15a8c4df91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:48.853315 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-4dea0e61-50fc-42ea-a6b0-af15a8c4df91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:48.854043 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9c5f7563-0fd8-47d3-878e-21ba2f4dfdbf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.219651 Oct 24 08:51:48.854441 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-4dea0e61-50fc-42ea-a6b0-af15a8c4df91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.226376 Oct 24 08:51:48.854613 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 65/138] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:48 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:48.855709 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 74/139] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:48 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:48.906351 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-962e91dc-85f4-4192-9611-5773f73af809 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:48.906951 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-962e91dc-85f4-4192-9611-5773f73af809 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:48.913106 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-962e91dc-85f4-4192-9611-5773f73af809 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:48.913594 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-962e91dc-85f4-4192-9611-5773f73af809 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:48.914071 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-962e91dc-85f4-4192-9611-5773f73af809 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:48.914734 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-962e91dc-85f4-4192-9611-5773f73af809 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:48.915185 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-962e91dc-85f4-4192-9611-5773f73af809 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:48.915608 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-962e91dc-85f4-4192-9611-5773f73af809 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:49.050391 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-962e91dc-85f4-4192-9611-5773f73af809 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:49.053784 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=100 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:49.058547 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-962e91dc-85f4-4192-9611-5773f73af809 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.156604 Oct 24 08:51:49.059113 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 66/140] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:48 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:49.315217 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-cba80533-0a97-4cc0-b1e1-7976b9888f15 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:49.315814 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-cba80533-0a97-4cc0-b1e1-7976b9888f15 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:49.321921 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cba80533-0a97-4cc0-b1e1-7976b9888f15 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:49.322509 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cba80533-0a97-4cc0-b1e1-7976b9888f15 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:49.323018 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cba80533-0a97-4cc0-b1e1-7976b9888f15 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:49.323799 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cba80533-0a97-4cc0-b1e1-7976b9888f15 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:49.324281 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cba80533-0a97-4cc0-b1e1-7976b9888f15 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:49.324701 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cba80533-0a97-4cc0-b1e1-7976b9888f15 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:49.388941 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cba80533-0a97-4cc0-b1e1-7976b9888f15 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:49.397807 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-cba80533-0a97-4cc0-b1e1-7976b9888f15 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1300 microversion: 2.1 time: 0.086361 Oct 24 08:51:49.398704 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 75/141] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:49 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1300 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:49.879792 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-555983e9-36d7-461c-9749-9c495eeb9423 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:49.880426 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-555983e9-36d7-461c-9749-9c495eeb9423 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:49.881751 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d63b3a9a-e4d3-46ca-8014-8cc449a77153 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:49.882357 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d63b3a9a-e4d3-46ca-8014-8cc449a77153 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:49.886846 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-555983e9-36d7-461c-9749-9c495eeb9423 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:49.887326 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-555983e9-36d7-461c-9749-9c495eeb9423 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:49.887757 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-555983e9-36d7-461c-9749-9c495eeb9423 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:49.889399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-555983e9-36d7-461c-9749-9c495eeb9423 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:49.889399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-555983e9-36d7-461c-9749-9c495eeb9423 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:49.889602 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d63b3a9a-e4d3-46ca-8014-8cc449a77153 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:49.889695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-555983e9-36d7-461c-9749-9c495eeb9423 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:49.889770 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d63b3a9a-e4d3-46ca-8014-8cc449a77153 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:49.891040 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d63b3a9a-e4d3-46ca-8014-8cc449a77153 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:49.891040 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d63b3a9a-e4d3-46ca-8014-8cc449a77153 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:49.891324 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d63b3a9a-e4d3-46ca-8014-8cc449a77153 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:49.891749 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d63b3a9a-e4d3-46ca-8014-8cc449a77153 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:50.098655 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d63b3a9a-e4d3-46ca-8014-8cc449a77153 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:50.102147 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-555983e9-36d7-461c-9749-9c495eeb9423 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:50.102907 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d63b3a9a-e4d3-46ca-8014-8cc449a77153 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:50.104537 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d63b3a9a-e4d3-46ca-8014-8cc449a77153 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:50.105705 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d63b3a9a-e4d3-46ca-8014-8cc449a77153 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.227767 Oct 24 08:51:50.106231 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 76/142] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:49 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:51:50.111533 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-555983e9-36d7-461c-9749-9c495eeb9423 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.234972 Oct 24 08:51:50.112340 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 67/143] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:49 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:50.122552 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1be55a53-9f95-43d3-9251-dc1efaac950e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:50.123236 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1be55a53-9f95-43d3-9251-dc1efaac950e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:50.129502 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1be55a53-9f95-43d3-9251-dc1efaac950e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:50.130029 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1be55a53-9f95-43d3-9251-dc1efaac950e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:50.130400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1be55a53-9f95-43d3-9251-dc1efaac950e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:50.131079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1be55a53-9f95-43d3-9251-dc1efaac950e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:50.131576 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1be55a53-9f95-43d3-9251-dc1efaac950e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:50.131989 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1be55a53-9f95-43d3-9251-dc1efaac950e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:50.267508 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1be55a53-9f95-43d3-9251-dc1efaac950e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:50.275197 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1be55a53-9f95-43d3-9251-dc1efaac950e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.167176 Oct 24 08:51:50.279011 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 77/144] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:50 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:50.420817 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-50f9dd97-754f-4766-b1c3-7dabcbc268bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:50.421934 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-50f9dd97-754f-4766-b1c3-7dabcbc268bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:50.428212 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-50f9dd97-754f-4766-b1c3-7dabcbc268bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:50.428701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-50f9dd97-754f-4766-b1c3-7dabcbc268bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:50.429137 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-50f9dd97-754f-4766-b1c3-7dabcbc268bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:50.429824 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-50f9dd97-754f-4766-b1c3-7dabcbc268bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:50.430276 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-50f9dd97-754f-4766-b1c3-7dabcbc268bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:50.430696 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-50f9dd97-754f-4766-b1c3-7dabcbc268bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:50.492573 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-50f9dd97-754f-4766-b1c3-7dabcbc268bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:50.500989 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-50f9dd97-754f-4766-b1c3-7dabcbc268bb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1300 microversion: 2.1 time: 0.084170 Oct 24 08:51:50.501545 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 68/145] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:50 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1300 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:51.131916 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d35af497-3c22-4d5b-8bb0-c77a5d314eae tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:51.132675 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d35af497-3c22-4d5b-8bb0-c77a5d314eae tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:51.133923 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a4bd7e3c-8e87-45b5-881c-8d807341a7d0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:51.134562 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-a4bd7e3c-8e87-45b5-881c-8d807341a7d0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:51.138921 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d35af497-3c22-4d5b-8bb0-c77a5d314eae tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:51.139404 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d35af497-3c22-4d5b-8bb0-c77a5d314eae tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:51.139845 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d35af497-3c22-4d5b-8bb0-c77a5d314eae tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:51.140691 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a4bd7e3c-8e87-45b5-881c-8d807341a7d0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:51.140833 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d35af497-3c22-4d5b-8bb0-c77a5d314eae tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:51.140994 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a4bd7e3c-8e87-45b5-881c-8d807341a7d0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:51.141094 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d35af497-3c22-4d5b-8bb0-c77a5d314eae tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:51.141412 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d35af497-3c22-4d5b-8bb0-c77a5d314eae tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:51.141500 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a4bd7e3c-8e87-45b5-881c-8d807341a7d0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:51.142123 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a4bd7e3c-8e87-45b5-881c-8d807341a7d0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:51.142579 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a4bd7e3c-8e87-45b5-881c-8d807341a7d0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:51.143005 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a4bd7e3c-8e87-45b5-881c-8d807341a7d0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:51.344277 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d35af497-3c22-4d5b-8bb0-c77a5d314eae tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:51.346894 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-a4bd7e3c-8e87-45b5-881c-8d807341a7d0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:51.354072 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d35af497-3c22-4d5b-8bb0-c77a5d314eae tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:51.355657 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-a4bd7e3c-8e87-45b5-881c-8d807341a7d0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.225374 Oct 24 08:51:51.355802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d35af497-3c22-4d5b-8bb0-c77a5d314eae tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:51.356272 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 69/146] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:51 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:51.356964 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d35af497-3c22-4d5b-8bb0-c77a5d314eae tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.229181 Oct 24 08:51:51.357521 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 78/147] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:51 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:51.372163 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2451688a-1b0b-4b4f-9a74-19373d2f3c70 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:51.372163 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2451688a-1b0b-4b4f-9a74-19373d2f3c70 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:51.372163 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2451688a-1b0b-4b4f-9a74-19373d2f3c70 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:51.372163 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2451688a-1b0b-4b4f-9a74-19373d2f3c70 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:51.372770 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2451688a-1b0b-4b4f-9a74-19373d2f3c70 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:51.372770 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2451688a-1b0b-4b4f-9a74-19373d2f3c70 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:51.372770 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2451688a-1b0b-4b4f-9a74-19373d2f3c70 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:51.372770 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2451688a-1b0b-4b4f-9a74-19373d2f3c70 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:51.505284 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2451688a-1b0b-4b4f-9a74-19373d2f3c70 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:51.512818 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2451688a-1b0b-4b4f-9a74-19373d2f3c70 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.155617 Oct 24 08:51:51.513380 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 70/148] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:51 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:51.524502 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7e5d7ceb-6eaf-4422-81a5-4772381caaa7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:51.525115 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7e5d7ceb-6eaf-4422-81a5-4772381caaa7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:51.531371 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e5d7ceb-6eaf-4422-81a5-4772381caaa7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:51.531844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e5d7ceb-6eaf-4422-81a5-4772381caaa7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:51.532304 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e5d7ceb-6eaf-4422-81a5-4772381caaa7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:51.532992 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e5d7ceb-6eaf-4422-81a5-4772381caaa7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:51.533442 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e5d7ceb-6eaf-4422-81a5-4772381caaa7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:51.533866 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e5d7ceb-6eaf-4422-81a5-4772381caaa7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:51.595274 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7e5d7ceb-6eaf-4422-81a5-4772381caaa7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:51.602833 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7e5d7ceb-6eaf-4422-81a5-4772381caaa7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1300 microversion: 2.1 time: 0.082150 Oct 24 08:51:51.603404 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 79/149] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:51 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1300 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:52.381075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-1f014bbe-3765-4657-9421-8765e96fd024 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:52.381075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-1f014bbe-3765-4657-9421-8765e96fd024 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:52.388116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1f014bbe-3765-4657-9421-8765e96fd024 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:52.388613 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1f014bbe-3765-4657-9421-8765e96fd024 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:52.389034 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1f014bbe-3765-4657-9421-8765e96fd024 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:52.389766 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1f014bbe-3765-4657-9421-8765e96fd024 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:52.390448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-9fa526f7-549d-4cee-bf6a-8ee77b3550af tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:52.390916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1f014bbe-3765-4657-9421-8765e96fd024 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:52.391113 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-9fa526f7-549d-4cee-bf6a-8ee77b3550af tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:52.392187 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1f014bbe-3765-4657-9421-8765e96fd024 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:52.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9fa526f7-549d-4cee-bf6a-8ee77b3550af tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:52.399082 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9fa526f7-549d-4cee-bf6a-8ee77b3550af tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:52.399590 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9fa526f7-549d-4cee-bf6a-8ee77b3550af tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:52.400733 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9fa526f7-549d-4cee-bf6a-8ee77b3550af tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:52.401195 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9fa526f7-549d-4cee-bf6a-8ee77b3550af tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:52.401709 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9fa526f7-549d-4cee-bf6a-8ee77b3550af tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:52.610392 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-1f014bbe-3765-4657-9421-8765e96fd024 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:52.613948 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-9fa526f7-549d-4cee-bf6a-8ee77b3550af tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:52.616557 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=100 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:52.619880 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-1f014bbe-3765-4657-9421-8765e96fd024 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.243860 Oct 24 08:51:52.633997 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 71/150] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:52 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:52.633997 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8a948ebb-38be-4cb1-9a41-aa1c40b0e377 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:52.633997 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8a948ebb-38be-4cb1-9a41-aa1c40b0e377 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:52.633997 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8a948ebb-38be-4cb1-9a41-aa1c40b0e377 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:52.633997 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8a948ebb-38be-4cb1-9a41-aa1c40b0e377 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:52.634493 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8a948ebb-38be-4cb1-9a41-aa1c40b0e377 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:52.634493 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8a948ebb-38be-4cb1-9a41-aa1c40b0e377 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:52.634493 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8a948ebb-38be-4cb1-9a41-aa1c40b0e377 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:52.634703 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8a948ebb-38be-4cb1-9a41-aa1c40b0e377 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:52.638714 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-9fa526f7-549d-4cee-bf6a-8ee77b3550af tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:52.641100 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-9fa526f7-549d-4cee-bf6a-8ee77b3550af tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:52.642237 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-9fa526f7-549d-4cee-bf6a-8ee77b3550af tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.258125 Oct 24 08:51:52.642796 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 80/151] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:52 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:52.648403 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-04681be4-dfce-4080-a823-2aa2dc6e47fb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:52.649819 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-04681be4-dfce-4080-a823-2aa2dc6e47fb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:52.657615 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-04681be4-dfce-4080-a823-2aa2dc6e47fb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:52.658333 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-04681be4-dfce-4080-a823-2aa2dc6e47fb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:52.658932 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-04681be4-dfce-4080-a823-2aa2dc6e47fb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:52.660456 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-04681be4-dfce-4080-a823-2aa2dc6e47fb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:52.660915 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-04681be4-dfce-4080-a823-2aa2dc6e47fb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:52.661348 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-04681be4-dfce-4080-a823-2aa2dc6e47fb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:52.744362 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-04681be4-dfce-4080-a823-2aa2dc6e47fb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:52.753129 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-04681be4-dfce-4080-a823-2aa2dc6e47fb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1300 microversion: 2.1 time: 0.109189 Oct 24 08:51:52.753706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 81/152] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:52 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1300 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:52.802800 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8a948ebb-38be-4cb1-9a41-aa1c40b0e377 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:52.810394 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8a948ebb-38be-4cb1-9a41-aa1c40b0e377 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.188865 Oct 24 08:51:52.811118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 72/153] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:52 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:53.644398 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-33a974a2-e396-457c-9e1e-7e8194568ad0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:53.644993 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-33a974a2-e396-457c-9e1e-7e8194568ad0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:53.651213 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-33a974a2-e396-457c-9e1e-7e8194568ad0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:53.651729 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-33a974a2-e396-457c-9e1e-7e8194568ad0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:53.652190 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-33a974a2-e396-457c-9e1e-7e8194568ad0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:53.652916 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-33a974a2-e396-457c-9e1e-7e8194568ad0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:53.653394 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-33a974a2-e396-457c-9e1e-7e8194568ad0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:53.653818 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-33a974a2-e396-457c-9e1e-7e8194568ad0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:53.668291 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6b0d0534-7867-4817-829d-ee8e01c28889 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:53.669782 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6b0d0534-7867-4817-829d-ee8e01c28889 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:53.677682 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b0d0534-7867-4817-829d-ee8e01c28889 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:53.678468 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b0d0534-7867-4817-829d-ee8e01c28889 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:53.679032 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b0d0534-7867-4817-829d-ee8e01c28889 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:53.679808 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b0d0534-7867-4817-829d-ee8e01c28889 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:53.680530 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b0d0534-7867-4817-829d-ee8e01c28889 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:53.680957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b0d0534-7867-4817-829d-ee8e01c28889 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:53.850144 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-33a974a2-e396-457c-9e1e-7e8194568ad0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:53.859364 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-33a974a2-e396-457c-9e1e-7e8194568ad0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.217609 Oct 24 08:51:53.867428 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 82/154] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:53 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:53.867428 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-61fd1d26-c8b2-4610-bfb7-0f5c271ccebf tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:53.867428 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-61fd1d26-c8b2-4610-bfb7-0f5c271ccebf tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:53.871931 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-61fd1d26-c8b2-4610-bfb7-0f5c271ccebf tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:53.872443 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-61fd1d26-c8b2-4610-bfb7-0f5c271ccebf tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:53.872873 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-61fd1d26-c8b2-4610-bfb7-0f5c271ccebf tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:53.873560 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-61fd1d26-c8b2-4610-bfb7-0f5c271ccebf tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:53.874668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-61fd1d26-c8b2-4610-bfb7-0f5c271ccebf tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:53.875926 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-61fd1d26-c8b2-4610-bfb7-0f5c271ccebf tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:53.880955 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6b0d0534-7867-4817-829d-ee8e01c28889 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:53.891280 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6b0d0534-7867-4817-829d-ee8e01c28889 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:53.892784 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6b0d0534-7867-4817-829d-ee8e01c28889 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:53.893903 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6b0d0534-7867-4817-829d-ee8e01c28889 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.230653 Oct 24 08:51:53.894458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 73/155] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:53 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:51:53.900515 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c8b72250-339d-4671-8c0c-e322ddb50e33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:53.901863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c8b72250-339d-4671-8c0c-e322ddb50e33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:53.912502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c8b72250-339d-4671-8c0c-e322ddb50e33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:53.913000 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c8b72250-339d-4671-8c0c-e322ddb50e33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:53.913441 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c8b72250-339d-4671-8c0c-e322ddb50e33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:53.914146 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c8b72250-339d-4671-8c0c-e322ddb50e33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:53.914611 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c8b72250-339d-4671-8c0c-e322ddb50e33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:53.915083 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c8b72250-339d-4671-8c0c-e322ddb50e33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:53.958879 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-61fd1d26-c8b2-4610-bfb7-0f5c271ccebf tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:53.968872 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-61fd1d26-c8b2-4610-bfb7-0f5c271ccebf tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1300 microversion: 2.1 time: 0.107202 Oct 24 08:51:53.969791 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 83/156] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:53 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1300 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:54.096862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c8b72250-339d-4671-8c0c-e322ddb50e33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:54.105383 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c8b72250-339d-4671-8c0c-e322ddb50e33 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.209673 Oct 24 08:51:54.105962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 74/157] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:53 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:54.681569 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-0bba4c31-31a1-4466-98d2-32b3745f0712 req-732f827f-13c7-4d61-af7e-64e57de9eb63 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fad848f1-177f-482d-a460-8a554066970c", "name": "network-vif-plugged", "status": "completed", "tag": "ac62796b-0699-4c3e-84f4-a4a808a0dd94"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:51:54.694916 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-0bba4c31-31a1-4466-98d2-32b3745f0712 req-732f827f-13c7-4d61-af7e-64e57de9eb63 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:54.695865 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-0bba4c31-31a1-4466-98d2-32b3745f0712 req-732f827f-13c7-4d61-af7e-64e57de9eb63 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:54.696533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-0bba4c31-31a1-4466-98d2-32b3745f0712 req-732f827f-13c7-4d61-af7e-64e57de9eb63 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:54.713495 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-0bba4c31-31a1-4466-98d2-32b3745f0712 req-732f827f-13c7-4d61-af7e-64e57de9eb63 service nova] Creating event network-vif-plugged:ac62796b-0699-4c3e-84f4-a4a808a0dd94 for instance fad848f1-177f-482d-a460-8a554066970c on hv-cinder-maste Oct 24 08:51:54.721442 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-0bba4c31-31a1-4466-98d2-32b3745f0712 req-732f827f-13c7-4d61-af7e-64e57de9eb63 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043801 Oct 24 08:51:54.724994 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 84/158] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:51:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:51:54.891209 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-59c64e1f-ff38-4a59-b4e0-1d1dc1868df2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:54.891826 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-59c64e1f-ff38-4a59-b4e0-1d1dc1868df2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:54.898146 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-59c64e1f-ff38-4a59-b4e0-1d1dc1868df2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:54.899440 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-59c64e1f-ff38-4a59-b4e0-1d1dc1868df2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:54.899878 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-59c64e1f-ff38-4a59-b4e0-1d1dc1868df2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:54.900627 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-59c64e1f-ff38-4a59-b4e0-1d1dc1868df2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:54.901219 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-59c64e1f-ff38-4a59-b4e0-1d1dc1868df2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:54.901792 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-59c64e1f-ff38-4a59-b4e0-1d1dc1868df2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:54.925398 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-199768c2-4fe8-424b-a9aa-2f3c2256f29d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:54.926113 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-199768c2-4fe8-424b-a9aa-2f3c2256f29d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:54.933367 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-199768c2-4fe8-424b-a9aa-2f3c2256f29d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:54.934124 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-199768c2-4fe8-424b-a9aa-2f3c2256f29d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:54.934729 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-199768c2-4fe8-424b-a9aa-2f3c2256f29d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:54.935753 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-199768c2-4fe8-424b-a9aa-2f3c2256f29d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:54.936661 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-199768c2-4fe8-424b-a9aa-2f3c2256f29d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:54.938160 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-199768c2-4fe8-424b-a9aa-2f3c2256f29d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:55.155954 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-59c64e1f-ff38-4a59-b4e0-1d1dc1868df2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:55.164651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-59c64e1f-ff38-4a59-b4e0-1d1dc1868df2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.277867 Oct 24 08:51:55.165797 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 75/159] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:54 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:55.170717 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4f321083-5e91-42ce-9064-7677267c2248 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:55.171852 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4f321083-5e91-42ce-9064-7677267c2248 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:55.188802 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4f321083-5e91-42ce-9064-7677267c2248 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:55.189875 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4f321083-5e91-42ce-9064-7677267c2248 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:55.190480 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4f321083-5e91-42ce-9064-7677267c2248 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:55.191401 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4f321083-5e91-42ce-9064-7677267c2248 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:55.192228 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4f321083-5e91-42ce-9064-7677267c2248 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:55.193360 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4f321083-5e91-42ce-9064-7677267c2248 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:55.197186 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-199768c2-4fe8-424b-a9aa-2f3c2256f29d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:55.214659 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-199768c2-4fe8-424b-a9aa-2f3c2256f29d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:55.216755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-199768c2-4fe8-424b-a9aa-2f3c2256f29d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:55.218667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-199768c2-4fe8-424b-a9aa-2f3c2256f29d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.298052 Oct 24 08:51:55.219640 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 85/160] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:54 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:55.226856 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7f6edaea-0e86-452f-86a1-0d3a3efd457f tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:55.227751 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7f6edaea-0e86-452f-86a1-0d3a3efd457f tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:55.237045 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7f6edaea-0e86-452f-86a1-0d3a3efd457f tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:55.237859 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7f6edaea-0e86-452f-86a1-0d3a3efd457f tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:55.238876 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7f6edaea-0e86-452f-86a1-0d3a3efd457f tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:55.239906 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7f6edaea-0e86-452f-86a1-0d3a3efd457f tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:55.241039 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7f6edaea-0e86-452f-86a1-0d3a3efd457f tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:55.241613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7f6edaea-0e86-452f-86a1-0d3a3efd457f tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:55.408065 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4f321083-5e91-42ce-9064-7677267c2248 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:55.419162 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4f321083-5e91-42ce-9064-7677267c2248 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.251485 Oct 24 08:51:55.419774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 76/161] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:55 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:55.458497 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7f6edaea-0e86-452f-86a1-0d3a3efd457f tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:55.469914 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7f6edaea-0e86-452f-86a1-0d3a3efd457f tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.248629 Oct 24 08:51:55.470762 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 86/162] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:55 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:56.192533 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7f9fbd00-24ea-4d12-8439-7de33ec99e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:56.193393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7f9fbd00-24ea-4d12-8439-7de33ec99e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:56.212663 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f9fbd00-24ea-4d12-8439-7de33ec99e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:56.213510 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f9fbd00-24ea-4d12-8439-7de33ec99e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:56.214089 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f9fbd00-24ea-4d12-8439-7de33ec99e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:56.215115 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f9fbd00-24ea-4d12-8439-7de33ec99e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:56.215719 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f9fbd00-24ea-4d12-8439-7de33ec99e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:56.216398 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f9fbd00-24ea-4d12-8439-7de33ec99e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:56.250599 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-2363a144-2b1d-4081-82b1-8862776de5e2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:56.251276 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-2363a144-2b1d-4081-82b1-8862776de5e2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:56.287556 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2363a144-2b1d-4081-82b1-8862776de5e2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:56.290813 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2363a144-2b1d-4081-82b1-8862776de5e2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:56.294830 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2363a144-2b1d-4081-82b1-8862776de5e2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:56.297159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2363a144-2b1d-4081-82b1-8862776de5e2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:56.299392 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2363a144-2b1d-4081-82b1-8862776de5e2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:56.300000 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2363a144-2b1d-4081-82b1-8862776de5e2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:56.621296 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7f9fbd00-24ea-4d12-8439-7de33ec99e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:56.625974 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-2363a144-2b1d-4081-82b1-8862776de5e2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:56.637382 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-2363a144-2b1d-4081-82b1-8862776de5e2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:56.642447 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-2363a144-2b1d-4081-82b1-8862776de5e2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:56.642607 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7f9fbd00-24ea-4d12-8439-7de33ec99e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1321 microversion: 2.1 time: 0.453739 Oct 24 08:51:56.642607 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 77/163] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:56 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1321 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:56.642772 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-2363a144-2b1d-4081-82b1-8862776de5e2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.397292 Oct 24 08:51:56.655281 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-66f7417d-0419-44e6-84b7-31754e296cac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:56.656368 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-66f7417d-0419-44e6-84b7-31754e296cac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:56.664694 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-66f7417d-0419-44e6-84b7-31754e296cac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:56.665482 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-66f7417d-0419-44e6-84b7-31754e296cac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:56.665920 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-66f7417d-0419-44e6-84b7-31754e296cac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:56.666628 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-66f7417d-0419-44e6-84b7-31754e296cac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:56.667108 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-66f7417d-0419-44e6-84b7-31754e296cac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:56.667690 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-66f7417d-0419-44e6-84b7-31754e296cac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:56.668735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 87/164] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:56 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:56.686238 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d99f6491-3d07-4ef5-9dda-a1b99eca292a tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:56.688269 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d99f6491-3d07-4ef5-9dda-a1b99eca292a tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:56.699088 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d99f6491-3d07-4ef5-9dda-a1b99eca292a tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:56.705450 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d99f6491-3d07-4ef5-9dda-a1b99eca292a tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:56.708285 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d99f6491-3d07-4ef5-9dda-a1b99eca292a tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:56.710305 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d99f6491-3d07-4ef5-9dda-a1b99eca292a tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:56.711701 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d99f6491-3d07-4ef5-9dda-a1b99eca292a tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:56.712521 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d99f6491-3d07-4ef5-9dda-a1b99eca292a tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:57.244336 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-66f7417d-0419-44e6-84b7-31754e296cac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:57.272584 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-66f7417d-0419-44e6-84b7-31754e296cac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.629176 Oct 24 08:51:57.273268 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 78/165] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:56 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 631 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:57.283954 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-b29893f6-4af6-49a2-b6a3-8fb663af9510 req-22eadafd-369d-468a-bab6-03f5674156cd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a85913b0-03e9-4510-9fc7-1514f34c95c1", "tag": "91e7f21e-e0a0-4b2a-9221-473c98847085"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:51:57.295535 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-b29893f6-4af6-49a2-b6a3-8fb663af9510 req-22eadafd-369d-468a-bab6-03f5674156cd service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:57.297102 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-b29893f6-4af6-49a2-b6a3-8fb663af9510 req-22eadafd-369d-468a-bab6-03f5674156cd service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:57.297537 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-b29893f6-4af6-49a2-b6a3-8fb663af9510 req-22eadafd-369d-468a-bab6-03f5674156cd service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:57.300911 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d99f6491-3d07-4ef5-9dda-a1b99eca292a tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:57.310086 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-b29893f6-4af6-49a2-b6a3-8fb663af9510 req-22eadafd-369d-468a-bab6-03f5674156cd service nova] Creating event network-changed:91e7f21e-e0a0-4b2a-9221-473c98847085 for instance a85913b0-03e9-4510-9fc7-1514f34c95c1 on hv-cinder-maste Oct 24 08:51:57.311522 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d99f6491-3d07-4ef5-9dda-a1b99eca292a tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.641612 Oct 24 08:51:57.319339 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 88/166] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:56 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 643 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:57.330504 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-b29893f6-4af6-49a2-b6a3-8fb663af9510 req-22eadafd-369d-468a-bab6-03f5674156cd service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055248 Oct 24 08:51:57.331154 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 79/167] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:51:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:51:57.665081 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a0beb544-5a08-4319-a8f9-7aa97e671b3d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:57.666285 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a0beb544-5a08-4319-a8f9-7aa97e671b3d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:57.673390 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0beb544-5a08-4319-a8f9-7aa97e671b3d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:57.674544 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0beb544-5a08-4319-a8f9-7aa97e671b3d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:57.675190 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0beb544-5a08-4319-a8f9-7aa97e671b3d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:57.676547 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0beb544-5a08-4319-a8f9-7aa97e671b3d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:57.677248 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0beb544-5a08-4319-a8f9-7aa97e671b3d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:57.678195 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0beb544-5a08-4319-a8f9-7aa97e671b3d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:57.701668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-41e1d59b-2eae-4cf3-930b-7f5e0c57539a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:57.703257 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-41e1d59b-2eae-4cf3-930b-7f5e0c57539a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:57.713404 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-41e1d59b-2eae-4cf3-930b-7f5e0c57539a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:57.714095 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-41e1d59b-2eae-4cf3-930b-7f5e0c57539a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:57.714373 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-41e1d59b-2eae-4cf3-930b-7f5e0c57539a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:57.715139 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-41e1d59b-2eae-4cf3-930b-7f5e0c57539a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:57.715790 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-41e1d59b-2eae-4cf3-930b-7f5e0c57539a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:57.716111 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-41e1d59b-2eae-4cf3-930b-7f5e0c57539a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:58.021667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a0beb544-5a08-4319-a8f9-7aa97e671b3d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:58.029509 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a0beb544-5a08-4319-a8f9-7aa97e671b3d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1516 microversion: 2.1 time: 0.369348 Oct 24 08:51:58.031076 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 89/168] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:57 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1516 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:58.059462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c4ba80ac-b72a-4d6a-a762-35917af874cb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:51:58.063491 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c4ba80ac-b72a-4d6a-a762-35917af874cb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:58.069406 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c4ba80ac-b72a-4d6a-a762-35917af874cb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:58.069987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c4ba80ac-b72a-4d6a-a762-35917af874cb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:58.070483 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c4ba80ac-b72a-4d6a-a762-35917af874cb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:58.071243 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c4ba80ac-b72a-4d6a-a762-35917af874cb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:58.071848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c4ba80ac-b72a-4d6a-a762-35917af874cb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:58.072489 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c4ba80ac-b72a-4d6a-a762-35917af874cb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:58.075726 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-41e1d59b-2eae-4cf3-930b-7f5e0c57539a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:58.085978 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-41e1d59b-2eae-4cf3-930b-7f5e0c57539a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:58.088277 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-41e1d59b-2eae-4cf3-930b-7f5e0c57539a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:58.089695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-41e1d59b-2eae-4cf3-930b-7f5e0c57539a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.392656 Oct 24 08:51:58.090781 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 80/169] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:57 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:51:58.136660 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-c4ba80ac-b72a-4d6a-a762-35917af874cb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lazy-loading 'flavor' on Instance uuid 0ec5fcd8-ee4b-4143-90be-991e1da8936e {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:51:58.284397 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c4ba80ac-b72a-4d6a-a762-35917af874cb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "POST /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e/action" status: 200 len: 32 microversion: 2.1 time: 0.230247 Oct 24 08:51:58.290176 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 90/170] 10.222.0.39 () {60 vars in 1278 bytes} [Mon Oct 24 08:51:58 2022] POST /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e/action => generated 32 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 24 08:51:58.296566 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-3c86d84e-d8d6-4613-941e-63115bb56084 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:58.297256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-3c86d84e-d8d6-4613-941e-63115bb56084 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:58.299332 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c3bc59b3-6b8d-4a5c-af8c-0d11ce4b2465 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:58.300943 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c3bc59b3-6b8d-4a5c-af8c-0d11ce4b2465 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:58.305842 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3c86d84e-d8d6-4613-941e-63115bb56084 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:58.310722 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c3bc59b3-6b8d-4a5c-af8c-0d11ce4b2465 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:58.311651 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c3bc59b3-6b8d-4a5c-af8c-0d11ce4b2465 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:58.312377 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c3bc59b3-6b8d-4a5c-af8c-0d11ce4b2465 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:58.312851 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3c86d84e-d8d6-4613-941e-63115bb56084 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:58.313310 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3c86d84e-d8d6-4613-941e-63115bb56084 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:58.313825 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c3bc59b3-6b8d-4a5c-af8c-0d11ce4b2465 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:58.314365 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3c86d84e-d8d6-4613-941e-63115bb56084 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:58.314759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c3bc59b3-6b8d-4a5c-af8c-0d11ce4b2465 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:58.314875 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3c86d84e-d8d6-4613-941e-63115bb56084 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:58.315307 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3c86d84e-d8d6-4613-941e-63115bb56084 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:58.315842 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c3bc59b3-6b8d-4a5c-af8c-0d11ce4b2465 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:58.641230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c3bc59b3-6b8d-4a5c-af8c-0d11ce4b2465 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:58.653379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c3bc59b3-6b8d-4a5c-af8c-0d11ce4b2465 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.360237 Oct 24 08:51:58.654804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 91/171] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:58 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:58.662116 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-4b64f8ac-79ca-4dd1-84fd-1baacb05e877 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:58.662964 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-4b64f8ac-79ca-4dd1-84fd-1baacb05e877 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:58.670055 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4b64f8ac-79ca-4dd1-84fd-1baacb05e877 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:58.671004 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4b64f8ac-79ca-4dd1-84fd-1baacb05e877 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:58.673232 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4b64f8ac-79ca-4dd1-84fd-1baacb05e877 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:58.674753 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4b64f8ac-79ca-4dd1-84fd-1baacb05e877 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:58.675499 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4b64f8ac-79ca-4dd1-84fd-1baacb05e877 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:58.676495 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4b64f8ac-79ca-4dd1-84fd-1baacb05e877 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:58.698334 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-3c86d84e-d8d6-4613-941e-63115bb56084 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:58.710353 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-3c86d84e-d8d6-4613-941e-63115bb56084 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.419272 Oct 24 08:51:58.710996 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 81/172] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:58 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:51:58.882054 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-4b64f8ac-79ca-4dd1-84fd-1baacb05e877 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:58.893603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-4b64f8ac-79ca-4dd1-84fd-1baacb05e877 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1314 microversion: 2.1 time: 0.235947 Oct 24 08:51:58.896680 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 92/173] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:58 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1314 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:51:59.117077 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9db6c388-0ef6-4ed3-abcb-8b75fcf3110d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:59.118108 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9db6c388-0ef6-4ed3-abcb-8b75fcf3110d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:59.124550 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9db6c388-0ef6-4ed3-abcb-8b75fcf3110d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:59.125036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9db6c388-0ef6-4ed3-abcb-8b75fcf3110d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:59.125639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9db6c388-0ef6-4ed3-abcb-8b75fcf3110d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:59.126343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9db6c388-0ef6-4ed3-abcb-8b75fcf3110d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:59.126793 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9db6c388-0ef6-4ed3-abcb-8b75fcf3110d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:59.127251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9db6c388-0ef6-4ed3-abcb-8b75fcf3110d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:59.268101 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9db6c388-0ef6-4ed3-abcb-8b75fcf3110d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:59.277525 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9db6c388-0ef6-4ed3-abcb-8b75fcf3110d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:59.279202 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9db6c388-0ef6-4ed3-abcb-8b75fcf3110d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:51:59.280948 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9db6c388-0ef6-4ed3-abcb-8b75fcf3110d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.167526 Oct 24 08:51:59.281551 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 82/174] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:51:59 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:51:59.677682 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8fd913b7-54a1-4707-aecf-a86bc9495d24 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:59.679126 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8fd913b7-54a1-4707-aecf-a86bc9495d24 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:59.690335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8fd913b7-54a1-4707-aecf-a86bc9495d24 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:59.691082 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8fd913b7-54a1-4707-aecf-a86bc9495d24 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:59.691678 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8fd913b7-54a1-4707-aecf-a86bc9495d24 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:59.693661 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8fd913b7-54a1-4707-aecf-a86bc9495d24 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:59.696007 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8fd913b7-54a1-4707-aecf-a86bc9495d24 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:59.696662 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8fd913b7-54a1-4707-aecf-a86bc9495d24 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:59.737151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-09348179-2602-42b3-a770-979d3f374231 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:51:59.738373 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-09348179-2602-42b3-a770-979d3f374231 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:51:59.756716 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-09348179-2602-42b3-a770-979d3f374231 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:59.762044 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-09348179-2602-42b3-a770-979d3f374231 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:59.765370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-09348179-2602-42b3-a770-979d3f374231 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:51:59.766432 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-09348179-2602-42b3-a770-979d3f374231 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:51:59.768758 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-09348179-2602-42b3-a770-979d3f374231 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:51:59.771996 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-09348179-2602-42b3-a770-979d3f374231 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:00.125003 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8fd913b7-54a1-4707-aecf-a86bc9495d24 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:00.135246 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8fd913b7-54a1-4707-aecf-a86bc9495d24 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.461313 Oct 24 08:52:00.139285 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 93/175] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:59 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:00.147441 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-12a740e9-959a-4e8a-b424-7962174ec7a0 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:00.148052 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-12a740e9-959a-4e8a-b424-7962174ec7a0 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:00.162743 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-12a740e9-959a-4e8a-b424-7962174ec7a0 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:00.165772 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-09348179-2602-42b3-a770-979d3f374231 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:00.175106 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-12a740e9-959a-4e8a-b424-7962174ec7a0 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:00.176836 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-12a740e9-959a-4e8a-b424-7962174ec7a0 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:00.177614 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-12a740e9-959a-4e8a-b424-7962174ec7a0 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:00.178007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-09348179-2602-42b3-a770-979d3f374231 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.444383 Oct 24 08:52:00.178914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 83/176] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:51:59 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:52:00.193138 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-12a740e9-959a-4e8a-b424-7962174ec7a0 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:00.193812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-12a740e9-959a-4e8a-b424-7962174ec7a0 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:00.309063 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-fb71aceb-7eca-434a-b53c-888118d41f15 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:00.309888 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-fb71aceb-7eca-434a-b53c-888118d41f15 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:00.321227 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fb71aceb-7eca-434a-b53c-888118d41f15 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:00.322347 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fb71aceb-7eca-434a-b53c-888118d41f15 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:00.322794 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fb71aceb-7eca-434a-b53c-888118d41f15 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:00.323507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fb71aceb-7eca-434a-b53c-888118d41f15 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:00.323965 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fb71aceb-7eca-434a-b53c-888118d41f15 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:00.324417 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fb71aceb-7eca-434a-b53c-888118d41f15 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:00.331046 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=100 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:52:00.362459 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-12a740e9-959a-4e8a-b424-7962174ec7a0 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:00.372250 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-12a740e9-959a-4e8a-b424-7962174ec7a0 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1501 microversion: 2.1 time: 0.231582 Oct 24 08:52:00.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 94/177] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:00 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1501 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:00.535069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fb71aceb-7eca-434a-b53c-888118d41f15 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:00.547948 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fb71aceb-7eca-434a-b53c-888118d41f15 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:00.549785 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fb71aceb-7eca-434a-b53c-888118d41f15 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:00.551988 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-fb71aceb-7eca-434a-b53c-888118d41f15 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.248122 Oct 24 08:52:00.554786 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 84/178] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:52:00 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:52:01.167298 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c77e545c-90c0-4f87-af65-592ebfc4faba tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:01.167897 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c77e545c-90c0-4f87-af65-592ebfc4faba tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:01.178153 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c77e545c-90c0-4f87-af65-592ebfc4faba tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:01.178881 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c77e545c-90c0-4f87-af65-592ebfc4faba tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:01.180538 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c77e545c-90c0-4f87-af65-592ebfc4faba tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:01.182477 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c77e545c-90c0-4f87-af65-592ebfc4faba tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:01.184641 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c77e545c-90c0-4f87-af65-592ebfc4faba tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:01.185096 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c77e545c-90c0-4f87-af65-592ebfc4faba tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:01.207388 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-24c1f92c-f281-4b39-b819-7a6e64ce7b8d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:01.207990 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-24c1f92c-f281-4b39-b819-7a6e64ce7b8d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:01.216869 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-24c1f92c-f281-4b39-b819-7a6e64ce7b8d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:01.220931 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-24c1f92c-f281-4b39-b819-7a6e64ce7b8d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:01.221642 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-24c1f92c-f281-4b39-b819-7a6e64ce7b8d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:01.223490 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-24c1f92c-f281-4b39-b819-7a6e64ce7b8d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:01.223965 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-24c1f92c-f281-4b39-b819-7a6e64ce7b8d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:01.225042 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-24c1f92c-f281-4b39-b819-7a6e64ce7b8d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:01.511278 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-24c1f92c-f281-4b39-b819-7a6e64ce7b8d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:01.521629 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-24c1f92c-f281-4b39-b819-7a6e64ce7b8d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.318657 Oct 24 08:52:01.525917 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 85/179] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:01 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:52:01.554422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c77e545c-90c0-4f87-af65-592ebfc4faba tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:01.565216 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c77e545c-90c0-4f87-af65-592ebfc4faba tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.401859 Oct 24 08:52:01.565885 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 95/180] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:01 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:01.585465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c6eb54ea-51c6-4382-ac2f-620188b97709 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:01.586248 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c6eb54ea-51c6-4382-ac2f-620188b97709 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:01.592934 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c6eb54ea-51c6-4382-ac2f-620188b97709 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:01.593538 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c6eb54ea-51c6-4382-ac2f-620188b97709 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:01.593966 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c6eb54ea-51c6-4382-ac2f-620188b97709 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:01.595205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c6eb54ea-51c6-4382-ac2f-620188b97709 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:01.595925 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c6eb54ea-51c6-4382-ac2f-620188b97709 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:01.597342 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c6eb54ea-51c6-4382-ac2f-620188b97709 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:01.780654 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c6eb54ea-51c6-4382-ac2f-620188b97709 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:01.790945 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c6eb54ea-51c6-4382-ac2f-620188b97709 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:01.793176 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c6eb54ea-51c6-4382-ac2f-620188b97709 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:01.794457 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c6eb54ea-51c6-4382-ac2f-620188b97709 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.213277 Oct 24 08:52:01.794994 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 86/181] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:52:01 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:52:02.549987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c7c7c563-001b-4a2e-b721-9892b8c47abc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:02.550571 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c7c7c563-001b-4a2e-b721-9892b8c47abc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:02.557205 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7c7c563-001b-4a2e-b721-9892b8c47abc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:02.557694 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7c7c563-001b-4a2e-b721-9892b8c47abc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:02.558120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7c7c563-001b-4a2e-b721-9892b8c47abc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:02.558817 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7c7c563-001b-4a2e-b721-9892b8c47abc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:02.559268 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7c7c563-001b-4a2e-b721-9892b8c47abc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:02.559685 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7c7c563-001b-4a2e-b721-9892b8c47abc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:02.563254 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:52:02.591533 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2b96db68-8b95-474d-8a42-d7639ed64d7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:02.592324 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2b96db68-8b95-474d-8a42-d7639ed64d7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:02.599647 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2b96db68-8b95-474d-8a42-d7639ed64d7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:02.600335 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2b96db68-8b95-474d-8a42-d7639ed64d7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:02.600886 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2b96db68-8b95-474d-8a42-d7639ed64d7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:02.601766 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2b96db68-8b95-474d-8a42-d7639ed64d7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:02.602265 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2b96db68-8b95-474d-8a42-d7639ed64d7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:02.602792 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2b96db68-8b95-474d-8a42-d7639ed64d7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:02.756091 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c7c7c563-001b-4a2e-b721-9892b8c47abc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:02.765501 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c7c7c563-001b-4a2e-b721-9892b8c47abc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.218935 Oct 24 08:52:02.781038 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 96/182] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:02 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:02.784277 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2b96db68-8b95-474d-8a42-d7639ed64d7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:02.792175 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2b96db68-8b95-474d-8a42-d7639ed64d7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.205375 Oct 24 08:52:02.792745 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 87/183] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:02 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:52:02.820693 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a3ebf1a0-8858-494c-93b1-32a50284e553 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:02.824468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a3ebf1a0-8858-494c-93b1-32a50284e553 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:02.831156 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a3ebf1a0-8858-494c-93b1-32a50284e553 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:02.831651 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a3ebf1a0-8858-494c-93b1-32a50284e553 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:02.832093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a3ebf1a0-8858-494c-93b1-32a50284e553 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:02.842296 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a3ebf1a0-8858-494c-93b1-32a50284e553 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:02.843884 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a3ebf1a0-8858-494c-93b1-32a50284e553 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:02.844071 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a3ebf1a0-8858-494c-93b1-32a50284e553 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:02.937434 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c2583b4e-8a68-4ea3-be1d-76ea388d4e29 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "60fc7d5e-921b-4156-af22-d50f26b4a9d3"}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:52:02.942379 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c2583b4e-8a68-4ea3-be1d-76ea388d4e29 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:02.950585 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c2583b4e-8a68-4ea3-be1d-76ea388d4e29 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:02.951394 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c2583b4e-8a68-4ea3-be1d-76ea388d4e29 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:02.952196 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c2583b4e-8a68-4ea3-be1d-76ea388d4e29 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:02.953551 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c2583b4e-8a68-4ea3-be1d-76ea388d4e29 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:02.954165 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c2583b4e-8a68-4ea3-be1d-76ea388d4e29 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:02.955076 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c2583b4e-8a68-4ea3-be1d-76ea388d4e29 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:03.069007 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a3ebf1a0-8858-494c-93b1-32a50284e553 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:03.080029 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a3ebf1a0-8858-494c-93b1-32a50284e553 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:03.082134 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a3ebf1a0-8858-494c-93b1-32a50284e553 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:03.091871 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a3ebf1a0-8858-494c-93b1-32a50284e553 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.274877 Oct 24 08:52:03.093904 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 97/184] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:52:02 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:52:03.564043 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c2583b4e-8a68-4ea3-be1d-76ea388d4e29 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "POST /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.630879 Oct 24 08:52:03.564877 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 88/185] 10.222.0.39 () {60 vars in 1323 bytes} [Mon Oct 24 08:52:02 2022] POST /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c/os-volume_attachments => generated 194 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:52:03.789784 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-fb973cc0-2d0b-4fec-99da-1ea0a8557150 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:03.790393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-fb973cc0-2d0b-4fec-99da-1ea0a8557150 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:03.796720 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fb973cc0-2d0b-4fec-99da-1ea0a8557150 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:03.797205 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fb973cc0-2d0b-4fec-99da-1ea0a8557150 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:03.797643 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fb973cc0-2d0b-4fec-99da-1ea0a8557150 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:03.798341 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fb973cc0-2d0b-4fec-99da-1ea0a8557150 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:03.798908 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fb973cc0-2d0b-4fec-99da-1ea0a8557150 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:03.799329 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fb973cc0-2d0b-4fec-99da-1ea0a8557150 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:03.816871 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-62a50dce-31ea-4d57-9fea-cbcaecfd5b7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:03.817778 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-62a50dce-31ea-4d57-9fea-cbcaecfd5b7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:03.825871 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62a50dce-31ea-4d57-9fea-cbcaecfd5b7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:03.826365 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62a50dce-31ea-4d57-9fea-cbcaecfd5b7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:03.827447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62a50dce-31ea-4d57-9fea-cbcaecfd5b7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:03.828420 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62a50dce-31ea-4d57-9fea-cbcaecfd5b7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:03.829015 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62a50dce-31ea-4d57-9fea-cbcaecfd5b7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:03.830002 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62a50dce-31ea-4d57-9fea-cbcaecfd5b7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:04.031568 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-fb973cc0-2d0b-4fec-99da-1ea0a8557150 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:04.044189 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-62a50dce-31ea-4d57-9fea-cbcaecfd5b7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:04.044783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-fb973cc0-2d0b-4fec-99da-1ea0a8557150 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.258544 Oct 24 08:52:04.046704 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 98/186] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:03 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:04.069411 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-62a50dce-31ea-4d57-9fea-cbcaecfd5b7d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.255148 Oct 24 08:52:04.072530 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 89/187] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:03 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:04.119769 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-932ec121-db61-44ca-a846-ba00358d8e84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:04.120807 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-932ec121-db61-44ca-a846-ba00358d8e84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:04.129158 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-932ec121-db61-44ca-a846-ba00358d8e84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:04.130100 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-932ec121-db61-44ca-a846-ba00358d8e84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:04.130714 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-932ec121-db61-44ca-a846-ba00358d8e84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:04.131476 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-932ec121-db61-44ca-a846-ba00358d8e84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:04.131934 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-932ec121-db61-44ca-a846-ba00358d8e84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:04.132389 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-932ec121-db61-44ca-a846-ba00358d8e84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:04.310439 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-932ec121-db61-44ca-a846-ba00358d8e84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:04.321063 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-932ec121-db61-44ca-a846-ba00358d8e84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:04.322837 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-932ec121-db61-44ca-a846-ba00358d8e84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:04.324273 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-932ec121-db61-44ca-a846-ba00358d8e84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.209106 Oct 24 08:52:04.325013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 99/188] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:52:04 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:52:05.068502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-dbc9ff00-7441-4bb3-9b71-5023d9f2d9c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:05.069312 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-dbc9ff00-7441-4bb3-9b71-5023d9f2d9c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:05.076178 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dbc9ff00-7441-4bb3-9b71-5023d9f2d9c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:05.077384 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dbc9ff00-7441-4bb3-9b71-5023d9f2d9c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:05.078077 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dbc9ff00-7441-4bb3-9b71-5023d9f2d9c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:05.079149 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dbc9ff00-7441-4bb3-9b71-5023d9f2d9c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:05.079666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dbc9ff00-7441-4bb3-9b71-5023d9f2d9c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:05.080159 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dbc9ff00-7441-4bb3-9b71-5023d9f2d9c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:05.093966 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-0144efd1-7e3d-42a5-8950-01e24a09221a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:05.094590 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-0144efd1-7e3d-42a5-8950-01e24a09221a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:05.101929 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0144efd1-7e3d-42a5-8950-01e24a09221a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:05.102564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0144efd1-7e3d-42a5-8950-01e24a09221a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:05.103006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0144efd1-7e3d-42a5-8950-01e24a09221a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:05.103711 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0144efd1-7e3d-42a5-8950-01e24a09221a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:05.104311 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0144efd1-7e3d-42a5-8950-01e24a09221a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:05.104747 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0144efd1-7e3d-42a5-8950-01e24a09221a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:05.297058 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-dbc9ff00-7441-4bb3-9b71-5023d9f2d9c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:05.304015 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0144efd1-7e3d-42a5-8950-01e24a09221a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:05.305660 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-dbc9ff00-7441-4bb3-9b71-5023d9f2d9c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.241053 Oct 24 08:52:05.312576 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-0144efd1-7e3d-42a5-8950-01e24a09221a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.223271 Oct 24 08:52:05.313281 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 100/189] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:05 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:52:05.320443 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 90/190] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:05 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:05.355415 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8a9b9857-1625-4b42-a9bc-db1bfa89b380 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:05.356279 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8a9b9857-1625-4b42-a9bc-db1bfa89b380 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:05.364918 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8a9b9857-1625-4b42-a9bc-db1bfa89b380 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:05.367082 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8a9b9857-1625-4b42-a9bc-db1bfa89b380 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:05.368876 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8a9b9857-1625-4b42-a9bc-db1bfa89b380 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:05.369189 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8a9b9857-1625-4b42-a9bc-db1bfa89b380 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:05.369709 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8a9b9857-1625-4b42-a9bc-db1bfa89b380 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:05.370137 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8a9b9857-1625-4b42-a9bc-db1bfa89b380 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:05.535861 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8a9b9857-1625-4b42-a9bc-db1bfa89b380 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:05.546228 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8a9b9857-1625-4b42-a9bc-db1bfa89b380 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:05.548270 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8a9b9857-1625-4b42-a9bc-db1bfa89b380 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:05.549894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8a9b9857-1625-4b42-a9bc-db1bfa89b380 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.200981 Oct 24 08:52:05.565274 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 101/191] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:52:05 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:52:06.330501 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7416126e-c7c8-411b-bf00-81714c044580 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:06.331559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7416126e-c7c8-411b-bf00-81714c044580 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:06.336417 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-adf1bb3a-f364-4909-b303-3af5636c99ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:06.337813 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-adf1bb3a-f364-4909-b303-3af5636c99ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:06.339271 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7416126e-c7c8-411b-bf00-81714c044580 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:06.339876 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7416126e-c7c8-411b-bf00-81714c044580 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:06.340445 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7416126e-c7c8-411b-bf00-81714c044580 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:06.341133 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7416126e-c7c8-411b-bf00-81714c044580 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:06.341731 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7416126e-c7c8-411b-bf00-81714c044580 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:06.343059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7416126e-c7c8-411b-bf00-81714c044580 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:06.346381 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-adf1bb3a-f364-4909-b303-3af5636c99ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:06.346994 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-adf1bb3a-f364-4909-b303-3af5636c99ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:06.347522 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-adf1bb3a-f364-4909-b303-3af5636c99ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:06.348652 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-adf1bb3a-f364-4909-b303-3af5636c99ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:06.349222 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-adf1bb3a-f364-4909-b303-3af5636c99ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:06.349714 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-adf1bb3a-f364-4909-b303-3af5636c99ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:06.589481 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7416126e-c7c8-411b-bf00-81714c044580 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:06.604696 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-adf1bb3a-f364-4909-b303-3af5636c99ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:06.609515 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7416126e-c7c8-411b-bf00-81714c044580 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.282522 Oct 24 08:52:06.610744 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 91/192] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:06 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:52:06.616557 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0d6c31ee-035d-4110-87d1-9fa39f66749c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:06.618367 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-0d6c31ee-035d-4110-87d1-9fa39f66749c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:06.626085 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d6c31ee-035d-4110-87d1-9fa39f66749c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:06.626810 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d6c31ee-035d-4110-87d1-9fa39f66749c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:06.627483 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d6c31ee-035d-4110-87d1-9fa39f66749c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:06.628759 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d6c31ee-035d-4110-87d1-9fa39f66749c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:06.629391 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d6c31ee-035d-4110-87d1-9fa39f66749c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:06.630098 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d6c31ee-035d-4110-87d1-9fa39f66749c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:06.633245 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-adf1bb3a-f364-4909-b303-3af5636c99ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.299785 Oct 24 08:52:06.633847 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 102/193] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:06 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:06.967113 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0d6c31ee-035d-4110-87d1-9fa39f66749c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:06.989530 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0d6c31ee-035d-4110-87d1-9fa39f66749c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:06.992011 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0d6c31ee-035d-4110-87d1-9fa39f66749c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:07.002895 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-0d6c31ee-035d-4110-87d1-9fa39f66749c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.390610 Oct 24 08:52:07.004215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 92/194] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:52:06 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:52:07.065680 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-ab1183a5-f4db-468a-9bd4-85d84c6a39df req-82b81ecd-0cbf-4b14-bcfd-83cc22c9ba83 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "71e106ba-ad54-4e9c-8e42-9477c8db8cf6", "name": "network-vif-plugged", "status": "completed", "tag": "b8754c97-b769-418c-b461-714a5809d621"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:52:07.084188 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-ab1183a5-f4db-468a-9bd4-85d84c6a39df req-82b81ecd-0cbf-4b14-bcfd-83cc22c9ba83 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:07.084897 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-ab1183a5-f4db-468a-9bd4-85d84c6a39df req-82b81ecd-0cbf-4b14-bcfd-83cc22c9ba83 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:07.085599 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-ab1183a5-f4db-468a-9bd4-85d84c6a39df req-82b81ecd-0cbf-4b14-bcfd-83cc22c9ba83 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:07.113076 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-ab1183a5-f4db-468a-9bd4-85d84c6a39df req-82b81ecd-0cbf-4b14-bcfd-83cc22c9ba83 service nova] Creating event network-vif-plugged:b8754c97-b769-418c-b461-714a5809d621 for instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 on hv-cinder-maste Oct 24 08:52:07.127300 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-ab1183a5-f4db-468a-9bd4-85d84c6a39df req-82b81ecd-0cbf-4b14-bcfd-83cc22c9ba83 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.066468 Oct 24 08:52:07.130812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 103/195] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:52:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:52:07.637406 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-fcc68886-76bc-4d9f-b4e6-890c6c10093a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:07.638077 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-fcc68886-76bc-4d9f-b4e6-890c6c10093a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:07.645900 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fcc68886-76bc-4d9f-b4e6-890c6c10093a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:07.646438 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fcc68886-76bc-4d9f-b4e6-890c6c10093a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:07.647108 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fcc68886-76bc-4d9f-b4e6-890c6c10093a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:07.647863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fcc68886-76bc-4d9f-b4e6-890c6c10093a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:07.648386 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fcc68886-76bc-4d9f-b4e6-890c6c10093a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:07.648814 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fcc68886-76bc-4d9f-b4e6-890c6c10093a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:07.659589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1d50e540-0911-43e1-bad2-f34cfa2e6913 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:07.661110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1d50e540-0911-43e1-bad2-f34cfa2e6913 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:07.668494 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1d50e540-0911-43e1-bad2-f34cfa2e6913 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:07.669022 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1d50e540-0911-43e1-bad2-f34cfa2e6913 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:07.669772 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1d50e540-0911-43e1-bad2-f34cfa2e6913 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:07.670516 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1d50e540-0911-43e1-bad2-f34cfa2e6913 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:07.671124 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1d50e540-0911-43e1-bad2-f34cfa2e6913 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:07.671701 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1d50e540-0911-43e1-bad2-f34cfa2e6913 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:07.894928 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fcc68886-76bc-4d9f-b4e6-890c6c10093a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:07.906287 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-fcc68886-76bc-4d9f-b4e6-890c6c10093a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.273404 Oct 24 08:52:07.906885 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 93/196] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:07 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:07.920994 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1d50e540-0911-43e1-bad2-f34cfa2e6913 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:07.931433 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1d50e540-0911-43e1-bad2-f34cfa2e6913 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.277345 Oct 24 08:52:07.944251 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 104/197] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:07 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:08.044759 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-30a6602c-bfd7-4c9e-9d51-385dede363df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:08.046150 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-30a6602c-bfd7-4c9e-9d51-385dede363df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:08.055999 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-30a6602c-bfd7-4c9e-9d51-385dede363df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:08.056543 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-30a6602c-bfd7-4c9e-9d51-385dede363df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:08.056974 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-30a6602c-bfd7-4c9e-9d51-385dede363df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:08.057877 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-30a6602c-bfd7-4c9e-9d51-385dede363df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:08.058327 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-30a6602c-bfd7-4c9e-9d51-385dede363df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:08.058743 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-30a6602c-bfd7-4c9e-9d51-385dede363df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:08.240652 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-30a6602c-bfd7-4c9e-9d51-385dede363df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:08.251589 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-30a6602c-bfd7-4c9e-9d51-385dede363df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:08.253836 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-30a6602c-bfd7-4c9e-9d51-385dede363df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:08.255916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-30a6602c-bfd7-4c9e-9d51-385dede363df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.219724 Oct 24 08:52:08.265752 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 94/198] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:52:08 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:52:08.932738 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-847325cf-683e-4272-804d-6b8403bc1d00 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:08.933306 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-847325cf-683e-4272-804d-6b8403bc1d00 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:08.940321 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-847325cf-683e-4272-804d-6b8403bc1d00 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:08.941001 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-847325cf-683e-4272-804d-6b8403bc1d00 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:08.941595 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-847325cf-683e-4272-804d-6b8403bc1d00 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:08.942549 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-847325cf-683e-4272-804d-6b8403bc1d00 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:08.943149 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-847325cf-683e-4272-804d-6b8403bc1d00 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:08.943820 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-847325cf-683e-4272-804d-6b8403bc1d00 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:08.961089 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ee6ea6db-3703-439f-bb88-a2bfe95c9567 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:08.962029 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ee6ea6db-3703-439f-bb88-a2bfe95c9567 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:08.974269 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee6ea6db-3703-439f-bb88-a2bfe95c9567 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:08.976929 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee6ea6db-3703-439f-bb88-a2bfe95c9567 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:08.977549 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee6ea6db-3703-439f-bb88-a2bfe95c9567 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:08.978463 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee6ea6db-3703-439f-bb88-a2bfe95c9567 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:08.980742 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee6ea6db-3703-439f-bb88-a2bfe95c9567 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:08.981371 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee6ea6db-3703-439f-bb88-a2bfe95c9567 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:09.228846 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ee6ea6db-3703-439f-bb88-a2bfe95c9567 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:09.236816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ee6ea6db-3703-439f-bb88-a2bfe95c9567 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.279580 Oct 24 08:52:09.237457 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 95/199] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:08 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:09.285989 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-68e21e0f-75d9-4c95-aa48-e5d55a2a405c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:09.287175 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-68e21e0f-75d9-4c95-aa48-e5d55a2a405c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:09.293931 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-68e21e0f-75d9-4c95-aa48-e5d55a2a405c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:09.294577 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-68e21e0f-75d9-4c95-aa48-e5d55a2a405c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:09.295188 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-68e21e0f-75d9-4c95-aa48-e5d55a2a405c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:09.296076 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-68e21e0f-75d9-4c95-aa48-e5d55a2a405c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:09.296698 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-68e21e0f-75d9-4c95-aa48-e5d55a2a405c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:09.297450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-68e21e0f-75d9-4c95-aa48-e5d55a2a405c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:09.348822 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-847325cf-683e-4272-804d-6b8403bc1d00 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:09.356597 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-847325cf-683e-4272-804d-6b8403bc1d00 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.430929 Oct 24 08:52:09.357476 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 105/200] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:08 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:09.493070 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-68e21e0f-75d9-4c95-aa48-e5d55a2a405c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:09.502239 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-68e21e0f-75d9-4c95-aa48-e5d55a2a405c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:09.503670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-68e21e0f-75d9-4c95-aa48-e5d55a2a405c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:09.505160 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-68e21e0f-75d9-4c95-aa48-e5d55a2a405c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1405 microversion: 2.20 time: 0.222994 Oct 24 08:52:09.505710 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 96/201] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:52:09 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1405 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:52:10.263322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-69f3cdbe-9a64-4c86-b0fb-5da4dbe88fa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:10.264452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-69f3cdbe-9a64-4c86-b0fb-5da4dbe88fa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:10.273494 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-69f3cdbe-9a64-4c86-b0fb-5da4dbe88fa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:10.276032 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-69f3cdbe-9a64-4c86-b0fb-5da4dbe88fa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:10.276516 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-69f3cdbe-9a64-4c86-b0fb-5da4dbe88fa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:10.277255 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-69f3cdbe-9a64-4c86-b0fb-5da4dbe88fa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:10.277720 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-69f3cdbe-9a64-4c86-b0fb-5da4dbe88fa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:10.278315 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-69f3cdbe-9a64-4c86-b0fb-5da4dbe88fa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:10.383472 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0fee9d31-fa02-4e46-984e-d062dfbf47b6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:10.385616 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-0fee9d31-fa02-4e46-984e-d062dfbf47b6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:10.402617 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fee9d31-fa02-4e46-984e-d062dfbf47b6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:10.404218 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fee9d31-fa02-4e46-984e-d062dfbf47b6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:10.406409 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fee9d31-fa02-4e46-984e-d062dfbf47b6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:10.407448 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fee9d31-fa02-4e46-984e-d062dfbf47b6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:10.408195 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fee9d31-fa02-4e46-984e-d062dfbf47b6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:10.408862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fee9d31-fa02-4e46-984e-d062dfbf47b6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:10.612389 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-69f3cdbe-9a64-4c86-b0fb-5da4dbe88fa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:10.622429 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-69f3cdbe-9a64-4c86-b0fb-5da4dbe88fa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.362967 Oct 24 08:52:10.624479 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 106/202] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:10 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:10.630656 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d964c98b-8661-4dc6-bf1e-10e58e730d84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:10.631379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d964c98b-8661-4dc6-bf1e-10e58e730d84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:10.638214 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d964c98b-8661-4dc6-bf1e-10e58e730d84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:10.638424 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d964c98b-8661-4dc6-bf1e-10e58e730d84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:10.639169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d964c98b-8661-4dc6-bf1e-10e58e730d84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:10.640463 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d964c98b-8661-4dc6-bf1e-10e58e730d84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:10.641201 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d964c98b-8661-4dc6-bf1e-10e58e730d84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:10.641885 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d964c98b-8661-4dc6-bf1e-10e58e730d84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:10.814110 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0fee9d31-fa02-4e46-984e-d062dfbf47b6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:10.834710 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-0fee9d31-fa02-4e46-984e-d062dfbf47b6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.457265 Oct 24 08:52:10.848834 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 97/203] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:10 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:52:10.883256 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d964c98b-8661-4dc6-bf1e-10e58e730d84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:10.907223 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d964c98b-8661-4dc6-bf1e-10e58e730d84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:10.912501 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d964c98b-8661-4dc6-bf1e-10e58e730d84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:10.915702 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d964c98b-8661-4dc6-bf1e-10e58e730d84 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1598 microversion: 2.20 time: 0.289539 Oct 24 08:52:10.916574 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 107/204] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:52:10 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1598 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:52:10.943881 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c0e66022-afe7-44ea-9489-c9230a7c67da tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:10.944394 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c0e66022-afe7-44ea-9489-c9230a7c67da tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:10.957038 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0e66022-afe7-44ea-9489-c9230a7c67da tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:10.958734 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0e66022-afe7-44ea-9489-c9230a7c67da tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:10.959598 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0e66022-afe7-44ea-9489-c9230a7c67da tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:10.961182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0e66022-afe7-44ea-9489-c9230a7c67da tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:10.961914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0e66022-afe7-44ea-9489-c9230a7c67da tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:10.963115 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0e66022-afe7-44ea-9489-c9230a7c67da tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:11.136884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c0e66022-afe7-44ea-9489-c9230a7c67da tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.196909 Oct 24 08:52:11.137469 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 98/205] 10.222.0.39 () {60 vars in 1316 bytes} [Mon Oct 24 08:52:10 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/os-interface => generated 285 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:52:11.656591 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b17e4c5a-cfd4-4292-950e-3359cba34467 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:11.657395 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b17e4c5a-cfd4-4292-950e-3359cba34467 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:11.674570 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b17e4c5a-cfd4-4292-950e-3359cba34467 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:11.675082 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b17e4c5a-cfd4-4292-950e-3359cba34467 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:11.675566 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b17e4c5a-cfd4-4292-950e-3359cba34467 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:11.676216 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b17e4c5a-cfd4-4292-950e-3359cba34467 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:11.676668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b17e4c5a-cfd4-4292-950e-3359cba34467 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:11.677083 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b17e4c5a-cfd4-4292-950e-3359cba34467 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:11.864047 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4924dc3c-3569-4647-b417-069dff6dc382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:11.865167 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4924dc3c-3569-4647-b417-069dff6dc382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:11.887528 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4924dc3c-3569-4647-b417-069dff6dc382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:11.888023 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4924dc3c-3569-4647-b417-069dff6dc382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:11.892898 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4924dc3c-3569-4647-b417-069dff6dc382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:11.893835 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4924dc3c-3569-4647-b417-069dff6dc382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:11.895040 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4924dc3c-3569-4647-b417-069dff6dc382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:11.896269 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4924dc3c-3569-4647-b417-069dff6dc382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:11.907868 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-b17e4c5a-cfd4-4292-950e-3359cba34467 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:11.917399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b17e4c5a-cfd4-4292-950e-3359cba34467 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.265748 Oct 24 08:52:11.918035 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 108/206] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:11 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:12.175750 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4924dc3c-3569-4647-b417-069dff6dc382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:12.196196 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:52:12.203116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4924dc3c-3569-4647-b417-069dff6dc382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.346220 Oct 24 08:52:12.207581 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 99/207] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:11 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:12.970651 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-40f22ee1-1aa9-421e-a7b9-d793df033bd6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:12.971461 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-40f22ee1-1aa9-421e-a7b9-d793df033bd6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:12.980427 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-40f22ee1-1aa9-421e-a7b9-d793df033bd6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:12.981322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-40f22ee1-1aa9-421e-a7b9-d793df033bd6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:12.982223 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-40f22ee1-1aa9-421e-a7b9-d793df033bd6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:12.985753 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-40f22ee1-1aa9-421e-a7b9-d793df033bd6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:12.987560 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-40f22ee1-1aa9-421e-a7b9-d793df033bd6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:12.988293 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-40f22ee1-1aa9-421e-a7b9-d793df033bd6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:13.238571 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-efa19206-79ab-4d92-9026-71d4e888603e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:13.240924 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-efa19206-79ab-4d92-9026-71d4e888603e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:13.283880 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-efa19206-79ab-4d92-9026-71d4e888603e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:13.285584 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-efa19206-79ab-4d92-9026-71d4e888603e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:13.288268 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-efa19206-79ab-4d92-9026-71d4e888603e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:13.289310 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-efa19206-79ab-4d92-9026-71d4e888603e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:13.290624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-efa19206-79ab-4d92-9026-71d4e888603e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:13.292863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-efa19206-79ab-4d92-9026-71d4e888603e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:13.479030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-40f22ee1-1aa9-421e-a7b9-d793df033bd6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:13.492960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-40f22ee1-1aa9-421e-a7b9-d793df033bd6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.530395 Oct 24 08:52:13.494012 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 109/208] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:12 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:13.501091 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-9f903500-10ee-4896-92cd-764bf3d9455c req-b16095e2-bb92-4de4-bd39-c6f513b9198c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "71e106ba-ad54-4e9c-8e42-9477c8db8cf6", "tag": "b8754c97-b769-418c-b461-714a5809d621"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:52:13.515323 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-9f903500-10ee-4896-92cd-764bf3d9455c req-b16095e2-bb92-4de4-bd39-c6f513b9198c service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:13.516222 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-9f903500-10ee-4896-92cd-764bf3d9455c req-b16095e2-bb92-4de4-bd39-c6f513b9198c service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:13.516835 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-9f903500-10ee-4896-92cd-764bf3d9455c req-b16095e2-bb92-4de4-bd39-c6f513b9198c service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:13.537124 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-9f903500-10ee-4896-92cd-764bf3d9455c req-b16095e2-bb92-4de4-bd39-c6f513b9198c service nova] Creating event network-changed:b8754c97-b769-418c-b461-714a5809d621 for instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 on hv-cinder-maste Oct 24 08:52:13.544104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-9f903500-10ee-4896-92cd-764bf3d9455c req-b16095e2-bb92-4de4-bd39-c6f513b9198c service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046698 Oct 24 08:52:13.545848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 110/209] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:52:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:52:13.628184 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-efa19206-79ab-4d92-9026-71d4e888603e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:13.639555 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-efa19206-79ab-4d92-9026-71d4e888603e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.410191 Oct 24 08:52:13.652711 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 100/210] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:13 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:14.521529 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ce8874ef-a01f-4ddc-b07e-01858e4b6579 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:14.522224 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ce8874ef-a01f-4ddc-b07e-01858e4b6579 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:14.530262 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ce8874ef-a01f-4ddc-b07e-01858e4b6579 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:14.530986 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ce8874ef-a01f-4ddc-b07e-01858e4b6579 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:14.531908 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ce8874ef-a01f-4ddc-b07e-01858e4b6579 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:14.532987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ce8874ef-a01f-4ddc-b07e-01858e4b6579 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:14.533789 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ce8874ef-a01f-4ddc-b07e-01858e4b6579 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:14.534365 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ce8874ef-a01f-4ddc-b07e-01858e4b6579 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:14.666143 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-100a5ebb-b6fe-4a9d-aa3f-6fbb99f811b7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:14.666939 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-100a5ebb-b6fe-4a9d-aa3f-6fbb99f811b7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:14.697073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-100a5ebb-b6fe-4a9d-aa3f-6fbb99f811b7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:14.699362 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-100a5ebb-b6fe-4a9d-aa3f-6fbb99f811b7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:14.700611 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-100a5ebb-b6fe-4a9d-aa3f-6fbb99f811b7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:14.705915 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-100a5ebb-b6fe-4a9d-aa3f-6fbb99f811b7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:14.706458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-100a5ebb-b6fe-4a9d-aa3f-6fbb99f811b7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:14.707909 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-100a5ebb-b6fe-4a9d-aa3f-6fbb99f811b7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:14.847108 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ce8874ef-a01f-4ddc-b07e-01858e4b6579 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:14.862378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ce8874ef-a01f-4ddc-b07e-01858e4b6579 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.342811 Oct 24 08:52:14.864655 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 111/211] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:14 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:14.907624 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-8cfe2ee4-5e7e-4fd1-b81d-778be7306770 req-c39aafca-1c49-4b60-a959-f65d7ae1596a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a85913b0-03e9-4510-9fc7-1514f34c95c1", "name": "network-vif-plugged", "status": "completed", "tag": "91e7f21e-e0a0-4b2a-9221-473c98847085"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:52:14.927105 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-8cfe2ee4-5e7e-4fd1-b81d-778be7306770 req-c39aafca-1c49-4b60-a959-f65d7ae1596a service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:14.927792 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-8cfe2ee4-5e7e-4fd1-b81d-778be7306770 req-c39aafca-1c49-4b60-a959-f65d7ae1596a service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:14.928413 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-8cfe2ee4-5e7e-4fd1-b81d-778be7306770 req-c39aafca-1c49-4b60-a959-f65d7ae1596a service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:14.948161 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-8cfe2ee4-5e7e-4fd1-b81d-778be7306770 req-c39aafca-1c49-4b60-a959-f65d7ae1596a service nova] Creating event network-vif-plugged:91e7f21e-e0a0-4b2a-9221-473c98847085 for instance a85913b0-03e9-4510-9fc7-1514f34c95c1 on hv-cinder-maste Oct 24 08:52:14.953783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-8cfe2ee4-5e7e-4fd1-b81d-778be7306770 req-c39aafca-1c49-4b60-a959-f65d7ae1596a service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050908 Oct 24 08:52:14.954895 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 112/212] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:52:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:52:14.974106 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-100a5ebb-b6fe-4a9d-aa3f-6fbb99f811b7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:14.983901 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-100a5ebb-b6fe-4a9d-aa3f-6fbb99f811b7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.322196 Oct 24 08:52:14.984861 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 101/213] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:14 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:15.889153 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c857541a-829f-480b-8b9e-3a5ebda408e4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:15.889868 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c857541a-829f-480b-8b9e-3a5ebda408e4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:15.905761 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c857541a-829f-480b-8b9e-3a5ebda408e4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:15.906650 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c857541a-829f-480b-8b9e-3a5ebda408e4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:15.907283 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c857541a-829f-480b-8b9e-3a5ebda408e4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:15.908844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c857541a-829f-480b-8b9e-3a5ebda408e4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:15.909531 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c857541a-829f-480b-8b9e-3a5ebda408e4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:15.910066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c857541a-829f-480b-8b9e-3a5ebda408e4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:16.008888 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9a310309-ef32-481f-b5fe-a318fa5be3fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:16.009501 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9a310309-ef32-481f-b5fe-a318fa5be3fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:16.018427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9a310309-ef32-481f-b5fe-a318fa5be3fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:16.019110 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9a310309-ef32-481f-b5fe-a318fa5be3fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:16.019873 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9a310309-ef32-481f-b5fe-a318fa5be3fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:16.020820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9a310309-ef32-481f-b5fe-a318fa5be3fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:16.021431 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9a310309-ef32-481f-b5fe-a318fa5be3fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:16.022012 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9a310309-ef32-481f-b5fe-a318fa5be3fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:16.323559 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c857541a-829f-480b-8b9e-3a5ebda408e4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:16.333240 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c857541a-829f-480b-8b9e-3a5ebda408e4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.449118 Oct 24 08:52:16.333923 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 113/214] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:15 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:16.350310 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9a310309-ef32-481f-b5fe-a318fa5be3fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:16.358495 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9a310309-ef32-481f-b5fe-a318fa5be3fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.354300 Oct 24 08:52:16.359061 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 102/215] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:16 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:52:17.357420 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c771504b-10e2-41d9-9752-e3b681fffe34 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:17.358377 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c771504b-10e2-41d9-9752-e3b681fffe34 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:17.363972 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=90 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:52:17.366003 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c771504b-10e2-41d9-9752-e3b681fffe34 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:17.366699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c771504b-10e2-41d9-9752-e3b681fffe34 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:17.367315 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c771504b-10e2-41d9-9752-e3b681fffe34 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:17.368281 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c771504b-10e2-41d9-9752-e3b681fffe34 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:17.369142 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c771504b-10e2-41d9-9752-e3b681fffe34 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:17.369641 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c771504b-10e2-41d9-9752-e3b681fffe34 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:17.386028 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-76f738a0-8046-45a3-9522-97624ec3b8b1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:17.386712 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-76f738a0-8046-45a3-9522-97624ec3b8b1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:17.397260 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76f738a0-8046-45a3-9522-97624ec3b8b1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:17.399750 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76f738a0-8046-45a3-9522-97624ec3b8b1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:17.400321 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76f738a0-8046-45a3-9522-97624ec3b8b1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:17.406840 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76f738a0-8046-45a3-9522-97624ec3b8b1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:17.407574 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76f738a0-8046-45a3-9522-97624ec3b8b1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:17.408090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76f738a0-8046-45a3-9522-97624ec3b8b1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:17.666159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c771504b-10e2-41d9-9752-e3b681fffe34 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:17.668380 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=100 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:52:17.674022 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-76f738a0-8046-45a3-9522-97624ec3b8b1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:17.676576 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c771504b-10e2-41d9-9752-e3b681fffe34 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.322879 Oct 24 08:52:17.677231 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 114/216] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:17 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:52:17.686226 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-76f738a0-8046-45a3-9522-97624ec3b8b1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.304161 Oct 24 08:52:17.686819 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 103/217] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:17 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:18.703468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-3be6e58e-3e5a-42c7-8805-ee3613f8b75c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:18.704705 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-3be6e58e-3e5a-42c7-8805-ee3613f8b75c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:18.712321 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4d8b9902-9dd5-46af-b066-2aec82c56484 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:18.712321 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4d8b9902-9dd5-46af-b066-2aec82c56484 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:18.713847 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=80 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:52:18.724379 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d8b9902-9dd5-46af-b066-2aec82c56484 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:18.724872 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d8b9902-9dd5-46af-b066-2aec82c56484 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:18.725303 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d8b9902-9dd5-46af-b066-2aec82c56484 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:18.725998 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d8b9902-9dd5-46af-b066-2aec82c56484 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:18.726448 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d8b9902-9dd5-46af-b066-2aec82c56484 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:18.726859 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d8b9902-9dd5-46af-b066-2aec82c56484 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:18.731922 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3be6e58e-3e5a-42c7-8805-ee3613f8b75c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:18.732434 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3be6e58e-3e5a-42c7-8805-ee3613f8b75c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:18.733692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3be6e58e-3e5a-42c7-8805-ee3613f8b75c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:18.734556 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3be6e58e-3e5a-42c7-8805-ee3613f8b75c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:18.735009 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3be6e58e-3e5a-42c7-8805-ee3613f8b75c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:18.735426 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3be6e58e-3e5a-42c7-8805-ee3613f8b75c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:18.941276 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-3be6e58e-3e5a-42c7-8805-ee3613f8b75c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:18.946596 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4d8b9902-9dd5-46af-b066-2aec82c56484 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:18.949541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-3be6e58e-3e5a-42c7-8805-ee3613f8b75c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.250869 Oct 24 08:52:18.950294 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 115/218] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:18 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:18.963164 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4d8b9902-9dd5-46af-b066-2aec82c56484 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.255641 Oct 24 08:52:18.964337 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 104/219] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:18 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:52:19.973639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-3d0fdad6-915f-48a2-a3b5-a8f76348a32f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:19.974821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-3d0fdad6-915f-48a2-a3b5-a8f76348a32f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:19.988300 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d0fdad6-915f-48a2-a3b5-a8f76348a32f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:19.988816 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d0fdad6-915f-48a2-a3b5-a8f76348a32f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:19.989274 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d0fdad6-915f-48a2-a3b5-a8f76348a32f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:19.990005 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d0fdad6-915f-48a2-a3b5-a8f76348a32f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:19.990734 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d0fdad6-915f-48a2-a3b5-a8f76348a32f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:19.991165 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d0fdad6-915f-48a2-a3b5-a8f76348a32f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:20.004450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-aee533bb-6f07-49d6-997b-de3124cee7b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:20.005814 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-aee533bb-6f07-49d6-997b-de3124cee7b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:20.022748 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aee533bb-6f07-49d6-997b-de3124cee7b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:20.024265 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aee533bb-6f07-49d6-997b-de3124cee7b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:20.024912 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aee533bb-6f07-49d6-997b-de3124cee7b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:20.026323 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aee533bb-6f07-49d6-997b-de3124cee7b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:20.027386 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aee533bb-6f07-49d6-997b-de3124cee7b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:20.028304 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aee533bb-6f07-49d6-997b-de3124cee7b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:20.229091 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-3d0fdad6-915f-48a2-a3b5-a8f76348a32f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:20.238062 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-3d0fdad6-915f-48a2-a3b5-a8f76348a32f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.268204 Oct 24 08:52:20.238887 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 116/220] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:19 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:20.286611 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-aee533bb-6f07-49d6-997b-de3124cee7b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:20.305366 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-aee533bb-6f07-49d6-997b-de3124cee7b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.318537 Oct 24 08:52:20.306368 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 105/221] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:19 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:21.284650 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-5ed1c589-eeb3-4e11-b9fb-d30b9c5cc20e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:21.285392 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-5ed1c589-eeb3-4e11-b9fb-d30b9c5cc20e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:21.312016 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5ed1c589-eeb3-4e11-b9fb-d30b9c5cc20e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:21.313532 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5ed1c589-eeb3-4e11-b9fb-d30b9c5cc20e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:21.313979 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5ed1c589-eeb3-4e11-b9fb-d30b9c5cc20e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:21.315018 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5ed1c589-eeb3-4e11-b9fb-d30b9c5cc20e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:21.321366 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5ed1c589-eeb3-4e11-b9fb-d30b9c5cc20e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:21.321613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5ed1c589-eeb3-4e11-b9fb-d30b9c5cc20e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:21.334340 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-3e923dd9-3098-4736-8337-e54f0e904801 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:21.335096 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-3e923dd9-3098-4736-8337-e54f0e904801 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:21.365291 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3e923dd9-3098-4736-8337-e54f0e904801 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:21.366049 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3e923dd9-3098-4736-8337-e54f0e904801 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:21.366494 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3e923dd9-3098-4736-8337-e54f0e904801 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:21.367227 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3e923dd9-3098-4736-8337-e54f0e904801 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:21.367681 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3e923dd9-3098-4736-8337-e54f0e904801 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:21.368097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3e923dd9-3098-4736-8337-e54f0e904801 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:21.646026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-5ed1c589-eeb3-4e11-b9fb-d30b9c5cc20e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:21.669246 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-5ed1c589-eeb3-4e11-b9fb-d30b9c5cc20e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.396033 Oct 24 08:52:21.670783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 117/222] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:21 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:21.695279 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-3e923dd9-3098-4736-8337-e54f0e904801 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:21.713759 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-3e923dd9-3098-4736-8337-e54f0e904801 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.386899 Oct 24 08:52:21.714930 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 106/223] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:21 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:52:22.662376 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c8cc01b4-5b3e-4099-b0d9-77f774b31246 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:22.663165 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c8cc01b4-5b3e-4099-b0d9-77f774b31246 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:22.669761 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c8cc01b4-5b3e-4099-b0d9-77f774b31246 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:22.670643 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c8cc01b4-5b3e-4099-b0d9-77f774b31246 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:22.671783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c8cc01b4-5b3e-4099-b0d9-77f774b31246 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:22.672726 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c8cc01b4-5b3e-4099-b0d9-77f774b31246 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:22.674206 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c8cc01b4-5b3e-4099-b0d9-77f774b31246 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:22.674818 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c8cc01b4-5b3e-4099-b0d9-77f774b31246 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:22.701994 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2df2039c-4da8-4d53-9439-fb04f651c1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:22.704964 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2df2039c-4da8-4d53-9439-fb04f651c1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:22.715780 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2df2039c-4da8-4d53-9439-fb04f651c1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:22.717772 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2df2039c-4da8-4d53-9439-fb04f651c1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:22.719020 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2df2039c-4da8-4d53-9439-fb04f651c1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:22.720675 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2df2039c-4da8-4d53-9439-fb04f651c1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:22.722245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2df2039c-4da8-4d53-9439-fb04f651c1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:22.724380 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2df2039c-4da8-4d53-9439-fb04f651c1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:22.737017 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c8cc01b4-5b3e-4099-b0d9-77f774b31246 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Going to try to terminate instance {{(pid=130395) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 08:52:22.887226 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c8cc01b4-5b3e-4099-b0d9-77f774b31246 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "DELETE /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 204 len: 0 microversion: 2.1 time: 0.228075 Oct 24 08:52:22.893377 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 118/224] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 08:52:22 2022] DELETE /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 08:52:22.907743 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-5e2ea89a-db12-49d2-8a86-10a33939d2c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:22.908775 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-5e2ea89a-db12-49d2-8a86-10a33939d2c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:22.917336 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e2ea89a-db12-49d2-8a86-10a33939d2c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:22.918732 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e2ea89a-db12-49d2-8a86-10a33939d2c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:22.919816 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e2ea89a-db12-49d2-8a86-10a33939d2c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:22.921438 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e2ea89a-db12-49d2-8a86-10a33939d2c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:22.922191 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e2ea89a-db12-49d2-8a86-10a33939d2c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:22.923056 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e2ea89a-db12-49d2-8a86-10a33939d2c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:23.022942 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2df2039c-4da8-4d53-9439-fb04f651c1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:23.040844 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2df2039c-4da8-4d53-9439-fb04f651c1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.342180 Oct 24 08:52:23.059050 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 107/225] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:22 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:52:23.088367 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b13f8822-8c62-4d95-8576-814435ffbf0e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:23.089357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b13f8822-8c62-4d95-8576-814435ffbf0e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:23.097997 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b13f8822-8c62-4d95-8576-814435ffbf0e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:23.098874 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b13f8822-8c62-4d95-8576-814435ffbf0e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:23.099521 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b13f8822-8c62-4d95-8576-814435ffbf0e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:23.100542 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b13f8822-8c62-4d95-8576-814435ffbf0e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:23.102222 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b13f8822-8c62-4d95-8576-814435ffbf0e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:23.102996 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b13f8822-8c62-4d95-8576-814435ffbf0e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:23.332817 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-5e2ea89a-db12-49d2-8a86-10a33939d2c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:23.342727 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-5e2ea89a-db12-49d2-8a86-10a33939d2c0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.445899 Oct 24 08:52:23.343462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 119/226] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:22 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:23.689628 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b13f8822-8c62-4d95-8576-814435ffbf0e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:23.700901 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b13f8822-8c62-4d95-8576-814435ffbf0e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1553 microversion: 2.1 time: 0.618651 Oct 24 08:52:23.702944 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 108/227] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:23 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1553 bytes in 643 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:24.092511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-410ba111-7774-4a5b-b29f-25cf02ccde3c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:24.093459 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-410ba111-7774-4a5b-b29f-25cf02ccde3c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:24.100617 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-410ba111-7774-4a5b-b29f-25cf02ccde3c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:24.101236 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-410ba111-7774-4a5b-b29f-25cf02ccde3c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:24.101749 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-410ba111-7774-4a5b-b29f-25cf02ccde3c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:24.102528 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-410ba111-7774-4a5b-b29f-25cf02ccde3c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:24.103118 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-410ba111-7774-4a5b-b29f-25cf02ccde3c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:24.103616 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-410ba111-7774-4a5b-b29f-25cf02ccde3c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:24.259343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-410ba111-7774-4a5b-b29f-25cf02ccde3c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:24.268553 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-410ba111-7774-4a5b-b29f-25cf02ccde3c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.181137 Oct 24 08:52:24.269135 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 120/228] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:24 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:24.372464 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ee4aceab-7350-4178-ba76-a968e8ae0ce3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:24.373431 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ee4aceab-7350-4178-ba76-a968e8ae0ce3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:24.380210 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee4aceab-7350-4178-ba76-a968e8ae0ce3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:24.380708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee4aceab-7350-4178-ba76-a968e8ae0ce3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:24.381152 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee4aceab-7350-4178-ba76-a968e8ae0ce3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:24.381855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee4aceab-7350-4178-ba76-a968e8ae0ce3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:24.382312 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee4aceab-7350-4178-ba76-a968e8ae0ce3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:24.382740 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee4aceab-7350-4178-ba76-a968e8ae0ce3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:24.541050 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ee4aceab-7350-4178-ba76-a968e8ae0ce3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:24.549840 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ee4aceab-7350-4178-ba76-a968e8ae0ce3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.182186 Oct 24 08:52:24.564681 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 109/229] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:24 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:24.741656 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-3d59a5c2-303f-4ffc-b502-c0d00fa2f81e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:24.743785 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-3d59a5c2-303f-4ffc-b502-c0d00fa2f81e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:24.762575 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d59a5c2-303f-4ffc-b502-c0d00fa2f81e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:24.763646 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d59a5c2-303f-4ffc-b502-c0d00fa2f81e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:24.764631 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d59a5c2-303f-4ffc-b502-c0d00fa2f81e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:24.766510 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d59a5c2-303f-4ffc-b502-c0d00fa2f81e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:24.770598 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d59a5c2-303f-4ffc-b502-c0d00fa2f81e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:24.772217 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d59a5c2-303f-4ffc-b502-c0d00fa2f81e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:24.953331 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-3d59a5c2-303f-4ffc-b502-c0d00fa2f81e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:24.972110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-3d59a5c2-303f-4ffc-b502-c0d00fa2f81e tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1553 microversion: 2.1 time: 0.241310 Oct 24 08:52:24.972416 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 121/230] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:24 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1553 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:25.293134 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-fb8cc370-28e2-489f-8f51-4580bb7e8e21 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:25.293888 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-fb8cc370-28e2-489f-8f51-4580bb7e8e21 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:25.315858 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fb8cc370-28e2-489f-8f51-4580bb7e8e21 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:25.316396 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fb8cc370-28e2-489f-8f51-4580bb7e8e21 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:25.316793 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fb8cc370-28e2-489f-8f51-4580bb7e8e21 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:25.319100 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fb8cc370-28e2-489f-8f51-4580bb7e8e21 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:25.319100 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fb8cc370-28e2-489f-8f51-4580bb7e8e21 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:25.319100 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fb8cc370-28e2-489f-8f51-4580bb7e8e21 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:25.496418 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fb8cc370-28e2-489f-8f51-4580bb7e8e21 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:25.514044 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-fb8cc370-28e2-489f-8f51-4580bb7e8e21 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.224281 Oct 24 08:52:25.515094 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 110/231] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:25 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:25.575252 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-5b1fb672-7fa9-4a37-8219-5040b555acd4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:25.577277 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-5b1fb672-7fa9-4a37-8219-5040b555acd4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:25.583686 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5b1fb672-7fa9-4a37-8219-5040b555acd4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:25.584196 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5b1fb672-7fa9-4a37-8219-5040b555acd4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:25.584949 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5b1fb672-7fa9-4a37-8219-5040b555acd4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:25.586006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5b1fb672-7fa9-4a37-8219-5040b555acd4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:25.586461 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5b1fb672-7fa9-4a37-8219-5040b555acd4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:25.586883 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5b1fb672-7fa9-4a37-8219-5040b555acd4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:25.758830 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-5b1fb672-7fa9-4a37-8219-5040b555acd4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:25.773146 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-5b1fb672-7fa9-4a37-8219-5040b555acd4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.201730 Oct 24 08:52:25.773765 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 122/232] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:25 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:26.005060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-313bfca7-7a7b-45ae-a1af-8ba3a6f87302 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:26.005060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-313bfca7-7a7b-45ae-a1af-8ba3a6f87302 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:26.010323 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-313bfca7-7a7b-45ae-a1af-8ba3a6f87302 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:26.010323 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-313bfca7-7a7b-45ae-a1af-8ba3a6f87302 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:26.011207 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-313bfca7-7a7b-45ae-a1af-8ba3a6f87302 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:26.011610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-313bfca7-7a7b-45ae-a1af-8ba3a6f87302 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:26.012316 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-313bfca7-7a7b-45ae-a1af-8ba3a6f87302 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:26.012907 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-313bfca7-7a7b-45ae-a1af-8ba3a6f87302 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:26.177258 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-313bfca7-7a7b-45ae-a1af-8ba3a6f87302 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:26.189422 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-313bfca7-7a7b-45ae-a1af-8ba3a6f87302 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1553 microversion: 2.1 time: 0.197576 Oct 24 08:52:26.189931 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 111/233] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:25 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1553 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:52:26.540173 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-014acc92-86d2-4096-983a-2b2c47da920f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:26.540846 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-014acc92-86d2-4096-983a-2b2c47da920f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:26.547974 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-014acc92-86d2-4096-983a-2b2c47da920f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:26.548564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-014acc92-86d2-4096-983a-2b2c47da920f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:26.549028 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-014acc92-86d2-4096-983a-2b2c47da920f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:26.550235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-014acc92-86d2-4096-983a-2b2c47da920f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:26.550827 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-014acc92-86d2-4096-983a-2b2c47da920f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:26.551131 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-014acc92-86d2-4096-983a-2b2c47da920f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:26.720280 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-014acc92-86d2-4096-983a-2b2c47da920f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:26.730459 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-014acc92-86d2-4096-983a-2b2c47da920f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1367 microversion: 2.1 time: 0.195145 Oct 24 08:52:26.731051 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 123/234] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:26 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1367 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:26.798070 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-41b0b530-29f8-426d-b08e-1cda0d947ad3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:26.798783 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-41b0b530-29f8-426d-b08e-1cda0d947ad3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:26.805617 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-41b0b530-29f8-426d-b08e-1cda0d947ad3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:26.806104 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-41b0b530-29f8-426d-b08e-1cda0d947ad3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:26.806531 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-41b0b530-29f8-426d-b08e-1cda0d947ad3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:26.807233 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-41b0b530-29f8-426d-b08e-1cda0d947ad3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:26.807678 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-41b0b530-29f8-426d-b08e-1cda0d947ad3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:26.808091 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-41b0b530-29f8-426d-b08e-1cda0d947ad3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:26.969527 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-41b0b530-29f8-426d-b08e-1cda0d947ad3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:26.989804 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-41b0b530-29f8-426d-b08e-1cda0d947ad3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.196351 Oct 24 08:52:26.991395 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 112/235] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:26 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:27.216403 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-6d231138-4409-45c0-8fb2-6ba59a00972b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:27.217095 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-6d231138-4409-45c0-8fb2-6ba59a00972b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:27.223590 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d231138-4409-45c0-8fb2-6ba59a00972b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:27.224085 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d231138-4409-45c0-8fb2-6ba59a00972b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:27.224565 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d231138-4409-45c0-8fb2-6ba59a00972b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:27.225294 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d231138-4409-45c0-8fb2-6ba59a00972b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:27.225748 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d231138-4409-45c0-8fb2-6ba59a00972b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:27.226171 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d231138-4409-45c0-8fb2-6ba59a00972b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:27.395572 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-6d231138-4409-45c0-8fb2-6ba59a00972b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:27.407899 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-6d231138-4409-45c0-8fb2-6ba59a00972b tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1553 microversion: 2.1 time: 0.195360 Oct 24 08:52:27.409031 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 124/236] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:27 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1553 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:27.566212 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-8881507f-dc4a-491a-a53d-76a22335cbaa req-4968d560-6332-459a-b2ad-d225b70268e8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fad848f1-177f-482d-a460-8a554066970c", "name": "network-vif-deleted", "tag": "ac62796b-0699-4c3e-84f4-a4a808a0dd94"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:52:27.579790 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-8881507f-dc4a-491a-a53d-76a22335cbaa req-4968d560-6332-459a-b2ad-d225b70268e8 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:27.580383 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-8881507f-dc4a-491a-a53d-76a22335cbaa req-4968d560-6332-459a-b2ad-d225b70268e8 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:27.580912 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-8881507f-dc4a-491a-a53d-76a22335cbaa req-4968d560-6332-459a-b2ad-d225b70268e8 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:27.610547 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-8881507f-dc4a-491a-a53d-76a22335cbaa req-4968d560-6332-459a-b2ad-d225b70268e8 service nova] Creating event network-vif-deleted:ac62796b-0699-4c3e-84f4-a4a808a0dd94 for instance fad848f1-177f-482d-a460-8a554066970c on hv-cinder-maste Oct 24 08:52:27.620505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-8881507f-dc4a-491a-a53d-76a22335cbaa req-4968d560-6332-459a-b2ad-d225b70268e8 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060842 Oct 24 08:52:27.621219 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 113/237] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:52:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:52:27.757925 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-5cb4465e-59ca-47dd-9a13-9c98dde53f28 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:27.758534 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-5cb4465e-59ca-47dd-9a13-9c98dde53f28 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:27.764952 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5cb4465e-59ca-47dd-9a13-9c98dde53f28 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:27.765447 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5cb4465e-59ca-47dd-9a13-9c98dde53f28 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:27.765891 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5cb4465e-59ca-47dd-9a13-9c98dde53f28 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:27.766607 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5cb4465e-59ca-47dd-9a13-9c98dde53f28 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:27.767071 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5cb4465e-59ca-47dd-9a13-9c98dde53f28 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:27.767503 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5cb4465e-59ca-47dd-9a13-9c98dde53f28 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:27.942187 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-5cb4465e-59ca-47dd-9a13-9c98dde53f28 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:27.950124 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-5cb4465e-59ca-47dd-9a13-9c98dde53f28 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1557 microversion: 2.1 time: 0.197363 Oct 24 08:52:27.950695 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 125/238] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:27 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1557 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:52:27.975897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-47ff7f69-8ead-4dea-a829-aec5b1ba528b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:27.976707 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-47ff7f69-8ead-4dea-a829-aec5b1ba528b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:27.983802 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47ff7f69-8ead-4dea-a829-aec5b1ba528b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:27.984336 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47ff7f69-8ead-4dea-a829-aec5b1ba528b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:27.984780 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47ff7f69-8ead-4dea-a829-aec5b1ba528b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:27.985502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47ff7f69-8ead-4dea-a829-aec5b1ba528b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:27.985994 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47ff7f69-8ead-4dea-a829-aec5b1ba528b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:27.986423 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47ff7f69-8ead-4dea-a829-aec5b1ba528b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:28.033414 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b7618187-de19-4b63-9f96-08262533eca0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:28.035599 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b7618187-de19-4b63-9f96-08262533eca0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:28.049214 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b7618187-de19-4b63-9f96-08262533eca0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:28.050294 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b7618187-de19-4b63-9f96-08262533eca0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:28.051839 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b7618187-de19-4b63-9f96-08262533eca0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:28.053764 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b7618187-de19-4b63-9f96-08262533eca0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:28.054275 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b7618187-de19-4b63-9f96-08262533eca0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:28.054830 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b7618187-de19-4b63-9f96-08262533eca0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:28.087047 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-47ff7f69-8ead-4dea-a829-aec5b1ba528b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.115987 Oct 24 08:52:28.087768 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 114/239] 10.222.0.39 () {58 vars in 1275 bytes} [Mon Oct 24 08:52:27 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1/os-interface => generated 284 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:52:28.244195 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-b7618187-de19-4b63-9f96-08262533eca0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:28.254159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b7618187-de19-4b63-9f96-08262533eca0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.230301 Oct 24 08:52:28.254909 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 126/240] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:28 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:28.436279 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-99a3f7e4-2931-4af7-beeb-dae84d3661bd tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:28.437120 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-99a3f7e4-2931-4af7-beeb-dae84d3661bd tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:28.444940 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99a3f7e4-2931-4af7-beeb-dae84d3661bd tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:28.445680 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99a3f7e4-2931-4af7-beeb-dae84d3661bd tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:28.446494 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99a3f7e4-2931-4af7-beeb-dae84d3661bd tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:28.447846 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99a3f7e4-2931-4af7-beeb-dae84d3661bd tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:28.448568 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99a3f7e4-2931-4af7-beeb-dae84d3661bd tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:28.449302 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99a3f7e4-2931-4af7-beeb-dae84d3661bd tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:28.506976 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99a3f7e4-2931-4af7-beeb-dae84d3661bd tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:28.507361 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99a3f7e4-2931-4af7-beeb-dae84d3661bd tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:28.507795 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99a3f7e4-2931-4af7-beeb-dae84d3661bd tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:28.508298 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.objects.instance [None req-99a3f7e4-2931-4af7-beeb-dae84d3661bd tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lazy-loading 'fault' on Instance uuid fad848f1-177f-482d-a460-8a554066970c {{(pid=130396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:52:28.559448 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-99a3f7e4-2931-4af7-beeb-dae84d3661bd tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '701e4507214f4634ad09e32e21749a35', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '25f1ebbfb9d042fab037dbe07a651913', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:28.572216 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-99a3f7e4-2931-4af7-beeb-dae84d3661bd tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 200 len: 1350 microversion: 2.1 time: 0.142528 Oct 24 08:52:28.573393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 115/241] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:28 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 1350 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:29.278703 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-42e944bd-ffc6-428c-a527-72c0888e4d15 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:29.280750 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-42e944bd-ffc6-428c-a527-72c0888e4d15 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:29.286158 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42e944bd-ffc6-428c-a527-72c0888e4d15 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:29.286655 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42e944bd-ffc6-428c-a527-72c0888e4d15 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:29.287491 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42e944bd-ffc6-428c-a527-72c0888e4d15 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:29.288229 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42e944bd-ffc6-428c-a527-72c0888e4d15 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:29.288691 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42e944bd-ffc6-428c-a527-72c0888e4d15 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:29.289120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42e944bd-ffc6-428c-a527-72c0888e4d15 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:29.429472 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-42e944bd-ffc6-428c-a527-72c0888e4d15 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:29.438633 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-42e944bd-ffc6-428c-a527-72c0888e4d15 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.164261 Oct 24 08:52:29.439212 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 127/242] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:29 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:29.611045 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9c60f8a3-ff8d-4997-ae0a-7aca63040774 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:29.611711 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9c60f8a3-ff8d-4997-ae0a-7aca63040774 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:29.619085 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c60f8a3-ff8d-4997-ae0a-7aca63040774 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:29.619701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c60f8a3-ff8d-4997-ae0a-7aca63040774 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:29.620142 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c60f8a3-ff8d-4997-ae0a-7aca63040774 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:29.620876 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c60f8a3-ff8d-4997-ae0a-7aca63040774 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:29.621348 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c60f8a3-ff8d-4997-ae0a-7aca63040774 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:29.621778 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c60f8a3-ff8d-4997-ae0a-7aca63040774 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:29.637561 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-d23252ee-abf5-4eeb-b658-e42212f34098 req-ef561e8c-5dc6-4a02-be82-4960ca9ac385 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a85913b0-03e9-4510-9fc7-1514f34c95c1", "tag": "91e7f21e-e0a0-4b2a-9221-473c98847085"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:52:29.637867 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-9c60f8a3-ff8d-4997-ae0a-7aca63040774 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] HTTP exception thrown: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:29.648428 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-d23252ee-abf5-4eeb-b658-e42212f34098 req-ef561e8c-5dc6-4a02-be82-4960ca9ac385 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:29.649026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-d23252ee-abf5-4eeb-b658-e42212f34098 req-ef561e8c-5dc6-4a02-be82-4960ca9ac385 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:29.649480 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-d23252ee-abf5-4eeb-b658-e42212f34098 req-ef561e8c-5dc6-4a02-be82-4960ca9ac385 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9c60f8a3-ff8d-4997-ae0a-7aca63040774 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Request method failure captured: Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c HTTP/1.1 Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlH2Ww0--UyBrdvyjHV67zYfHhPS5St4Ugg01Ey7L5caL19-aayhsoOdc6acFwUvlcmu7V0CogR8aTXpYTN7RyYdCXoYZhkj7VfTWdYtKDCWOnUXjChbgUwWTfcqrPE9qhiffyTaSpxUHa8ndw4UV9sFNOAhfR9KCyHI3156hF1tlgg Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 25f1ebbfb9d042fab037dbe07a651913 Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-DeleteServersTestJSON-116224961 Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 08:52:29.661060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 08:52:29.664056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/25f1ebbfb9d042fab037dbe07a651913"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_25f1ebbfb9d042fab037dbe07a651913", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/25f1ebbfb9d042fab037dbe07a651913"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/25f1ebbfb9d042fab037dbe07a651913"}]}] Oct 24 08:52:29.664056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-DeleteServersTestJSON-116224961 Oct 24 08:52:29.664056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 25f1ebbfb9d042fab037dbe07a651913 Oct 24 08:52:29.664056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-DeleteServersTestJSON-116224961 Oct 24 08:52:29.664056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-DeleteServersTestJSON-116224961-project Oct 24 08:52:29.664056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 08:52:29.664056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 08:52:29.664056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 701e4507214f4634ad09e32e21749a35 Oct 24 08:52:29.664056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-DeleteServersTestJSON-116224961-project Oct 24 08:52:29.664056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 08:52:29.664056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:29.664056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': 'fad848f1-177f-482d-a460-8a554066970c'} Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:52:29.665639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:29.668097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:52:29.671222 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9c60f8a3-ff8d-4997-ae0a-7aca63040774 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Returning 404 to user: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:52:29.671222 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9c60f8a3-ff8d-4997-ae0a-7aca63040774 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 404 len: 111 microversion: 2.1 time: 0.059035 Oct 24 08:52:29.671384 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-d23252ee-abf5-4eeb-b658-e42212f34098 req-ef561e8c-5dc6-4a02-be82-4960ca9ac385 service nova] Creating event network-changed:91e7f21e-e0a0-4b2a-9221-473c98847085 for instance a85913b0-03e9-4510-9fc7-1514f34c95c1 on hv-cinder-maste Oct 24 08:52:29.671475 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 116/243] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:29 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 08:52:29.673929 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-d23252ee-abf5-4eeb-b658-e42212f34098 req-ef561e8c-5dc6-4a02-be82-4960ca9ac385 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039768 Oct 24 08:52:29.673929 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 128/244] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:52:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:52:29.805813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-28706120-5f6f-4fb3-91f5-4a65b7f45355 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:29.808036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-28706120-5f6f-4fb3-91f5-4a65b7f45355 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:29.814991 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-28706120-5f6f-4fb3-91f5-4a65b7f45355 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:29.815488 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-28706120-5f6f-4fb3-91f5-4a65b7f45355 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:29.816228 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-28706120-5f6f-4fb3-91f5-4a65b7f45355 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:29.817075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-28706120-5f6f-4fb3-91f5-4a65b7f45355 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:29.817527 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-28706120-5f6f-4fb3-91f5-4a65b7f45355 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:29.818152 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-28706120-5f6f-4fb3-91f5-4a65b7f45355 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:29.836739 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-28706120-5f6f-4fb3-91f5-4a65b7f45355 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] HTTP exception thrown: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-28706120-5f6f-4fb3-91f5-4a65b7f45355 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Request method failure captured: Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c/os-volume_attachments HTTP/1.1 Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlH2Ww0--UyBrdvyjHV67zYfHhPS5St4Ugg01Ey7L5caL19-aayhsoOdc6acFwUvlcmu7V0CogR8aTXpYTN7RyYdCXoYZhkj7VfTWdYtKDCWOnUXjChbgUwWTfcqrPE9qhiffyTaSpxUHa8ndw4UV9sFNOAhfR9KCyHI3156hF1tlgg Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 25f1ebbfb9d042fab037dbe07a651913 Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-DeleteServersTestJSON-116224961 Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 08:52:29.838774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 08:52:29.841949 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/25f1ebbfb9d042fab037dbe07a651913"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_25f1ebbfb9d042fab037dbe07a651913", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/25f1ebbfb9d042fab037dbe07a651913"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/25f1ebbfb9d042fab037dbe07a651913"}]}] Oct 24 08:52:29.841949 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-DeleteServersTestJSON-116224961 Oct 24 08:52:29.841949 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 25f1ebbfb9d042fab037dbe07a651913 Oct 24 08:52:29.841949 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-DeleteServersTestJSON-116224961 Oct 24 08:52:29.841949 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-DeleteServersTestJSON-116224961-project Oct 24 08:52:29.841949 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 08:52:29.841949 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 08:52:29.841949 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 701e4507214f4634ad09e32e21749a35 Oct 24 08:52:29.841949 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-DeleteServersTestJSON-116224961-project Oct 24 08:52:29.841949 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 08:52:29.841949 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:29.841949 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'server_id': 'fad848f1-177f-482d-a460-8a554066970c'} Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:52:29.843387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:29.844776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:52:29.847251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-28706120-5f6f-4fb3-91f5-4a65b7f45355 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Returning 404 to user: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:52:29.848305 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-28706120-5f6f-4fb3-91f5-4a65b7f45355 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.046492 Oct 24 08:52:29.853810 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 117/245] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 08:52:29 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c/os-volume_attachments => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 08:52:30.463572 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-0ffd82b5-76bb-45dd-a32a-23271969826e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:30.464687 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-0ffd82b5-76bb-45dd-a32a-23271969826e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:30.471121 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0ffd82b5-76bb-45dd-a32a-23271969826e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:30.471949 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0ffd82b5-76bb-45dd-a32a-23271969826e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:30.472440 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0ffd82b5-76bb-45dd-a32a-23271969826e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:30.473304 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0ffd82b5-76bb-45dd-a32a-23271969826e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:30.473945 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0ffd82b5-76bb-45dd-a32a-23271969826e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:30.475679 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0ffd82b5-76bb-45dd-a32a-23271969826e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:30.650468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0ffd82b5-76bb-45dd-a32a-23271969826e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:30.661768 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-0ffd82b5-76bb-45dd-a32a-23271969826e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.202605 Oct 24 08:52:30.664838 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 129/246] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:30 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:31.134318 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-086ee2cc-ca52-4916-a71f-b857b7542d72 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:31.135035 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-086ee2cc-ca52-4916-a71f-b857b7542d72 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:31.142355 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-086ee2cc-ca52-4916-a71f-b857b7542d72 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:31.142864 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-086ee2cc-ca52-4916-a71f-b857b7542d72 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:31.143269 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-086ee2cc-ca52-4916-a71f-b857b7542d72 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:31.143966 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-086ee2cc-ca52-4916-a71f-b857b7542d72 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:31.144523 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-086ee2cc-ca52-4916-a71f-b857b7542d72 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:31.145146 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-086ee2cc-ca52-4916-a71f-b857b7542d72 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:31.160212 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-086ee2cc-ca52-4916-a71f-b857b7542d72 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] HTTP exception thrown: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-086ee2cc-ca52-4916-a71f-b857b7542d72 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Request method failure captured: Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: DELETE /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c HTTP/1.1 Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlH2Ww0--UyBrdvyjHV67zYfHhPS5St4Ugg01Ey7L5caL19-aayhsoOdc6acFwUvlcmu7V0CogR8aTXpYTN7RyYdCXoYZhkj7VfTWdYtKDCWOnUXjChbgUwWTfcqrPE9qhiffyTaSpxUHa8ndw4UV9sFNOAhfR9KCyHI3156hF1tlgg Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 25f1ebbfb9d042fab037dbe07a651913 Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-DeleteServersTestJSON-116224961 Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 08:52:31.161972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 08:52:31.163428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/25f1ebbfb9d042fab037dbe07a651913"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_25f1ebbfb9d042fab037dbe07a651913", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/25f1ebbfb9d042fab037dbe07a651913"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/25f1ebbfb9d042fab037dbe07a651913"}]}] Oct 24 08:52:31.163428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-DeleteServersTestJSON-116224961 Oct 24 08:52:31.163428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 25f1ebbfb9d042fab037dbe07a651913 Oct 24 08:52:31.163428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-DeleteServersTestJSON-116224961 Oct 24 08:52:31.163428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-DeleteServersTestJSON-116224961-project Oct 24 08:52:31.163428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 08:52:31.163428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 08:52:31.163428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 701e4507214f4634ad09e32e21749a35 Oct 24 08:52:31.163428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-DeleteServersTestJSON-116224961-project Oct 24 08:52:31.163428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 08:52:31.163428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:31.163428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': 'fad848f1-177f-482d-a460-8a554066970c'} Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:52:31.167427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:31.168897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:52:31.170246 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-086ee2cc-ca52-4916-a71f-b857b7542d72 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Returning 404 to user: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:52:31.170246 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-086ee2cc-ca52-4916-a71f-b857b7542d72 tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "DELETE /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 404 len: 111 microversion: 2.1 time: 0.035440 Oct 24 08:52:31.170246 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 118/247] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 08:52:31 2022] DELETE /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 24 08:52:31.172765 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-9a4fb20d-885e-4d3a-a7a2-105e042e654d tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:31.173500 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-9a4fb20d-885e-4d3a-a7a2-105e042e654d tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] [instance: fad848f1-177f-482d-a460-8a554066970c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:31.179461 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9a4fb20d-885e-4d3a-a7a2-105e042e654d tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:31.179955 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9a4fb20d-885e-4d3a-a7a2-105e042e654d tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:31.180453 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9a4fb20d-885e-4d3a-a7a2-105e042e654d tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:31.181226 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9a4fb20d-885e-4d3a-a7a2-105e042e654d tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:31.181714 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9a4fb20d-885e-4d3a-a7a2-105e042e654d tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:31.182145 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9a4fb20d-885e-4d3a-a7a2-105e042e654d tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:31.197010 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-9a4fb20d-885e-4d3a-a7a2-105e042e654d tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] HTTP exception thrown: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-9a4fb20d-885e-4d3a-a7a2-105e042e654d tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Request method failure captured: Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c HTTP/1.1 Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlH2Ww0--UyBrdvyjHV67zYfHhPS5St4Ugg01Ey7L5caL19-aayhsoOdc6acFwUvlcmu7V0CogR8aTXpYTN7RyYdCXoYZhkj7VfTWdYtKDCWOnUXjChbgUwWTfcqrPE9qhiffyTaSpxUHa8ndw4UV9sFNOAhfR9KCyHI3156hF1tlgg Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: 25f1ebbfb9d042fab037dbe07a651913 Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-DeleteServersTestJSON-116224961 Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 08:52:31.202511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 08:52:31.205207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/25f1ebbfb9d042fab037dbe07a651913"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_25f1ebbfb9d042fab037dbe07a651913", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/25f1ebbfb9d042fab037dbe07a651913"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/25f1ebbfb9d042fab037dbe07a651913"}]}] Oct 24 08:52:31.205207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-DeleteServersTestJSON-116224961 Oct 24 08:52:31.205207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: 25f1ebbfb9d042fab037dbe07a651913 Oct 24 08:52:31.205207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-DeleteServersTestJSON-116224961 Oct 24 08:52:31.205207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-DeleteServersTestJSON-116224961-project Oct 24 08:52:31.205207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 08:52:31.205207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 08:52:31.205207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: 701e4507214f4634ad09e32e21749a35 Oct 24 08:52:31.205207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-DeleteServersTestJSON-116224961-project Oct 24 08:52:31.205207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 08:52:31.205207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:31.205207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': 'fad848f1-177f-482d-a460-8a554066970c'} Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:52:31.206783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. Oct 24 08:52:31.210235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:52:31.213113 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-9a4fb20d-885e-4d3a-a7a2-105e042e654d tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] Returning 404 to user: Instance fad848f1-177f-482d-a460-8a554066970c could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:52:31.213113 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-9a4fb20d-885e-4d3a-a7a2-105e042e654d tempest-DeleteServersTestJSON-116224961 tempest-DeleteServersTestJSON-116224961-project] 10.222.0.39 "GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c" status: 404 len: 111 microversion: 2.1 time: 0.035580 Oct 24 08:52:31.213113 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 130/248] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:31 2022] GET /compute/v2.1/servers/fad848f1-177f-482d-a460-8a554066970c => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 08:52:31.690534 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7f713a2f-9831-4f8f-9252-e3b2c6f61dd2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:31.691263 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7f713a2f-9831-4f8f-9252-e3b2c6f61dd2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:31.697821 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f713a2f-9831-4f8f-9252-e3b2c6f61dd2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:31.698310 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f713a2f-9831-4f8f-9252-e3b2c6f61dd2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:31.699289 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f713a2f-9831-4f8f-9252-e3b2c6f61dd2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:31.700024 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f713a2f-9831-4f8f-9252-e3b2c6f61dd2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:31.700515 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f713a2f-9831-4f8f-9252-e3b2c6f61dd2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:31.701366 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f713a2f-9831-4f8f-9252-e3b2c6f61dd2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:31.706147 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=100 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:52:31.997811 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7f713a2f-9831-4f8f-9252-e3b2c6f61dd2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:32.006934 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7f713a2f-9831-4f8f-9252-e3b2c6f61dd2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.323557 Oct 24 08:52:32.007841 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 119/249] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:31 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:33.057843 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-917e87a5-390d-4744-b5a2-f36299350de3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:33.058972 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-917e87a5-390d-4744-b5a2-f36299350de3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:33.076832 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-917e87a5-390d-4744-b5a2-f36299350de3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:33.078648 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-917e87a5-390d-4744-b5a2-f36299350de3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:33.080471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-917e87a5-390d-4744-b5a2-f36299350de3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:33.082003 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-917e87a5-390d-4744-b5a2-f36299350de3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:33.083071 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-917e87a5-390d-4744-b5a2-f36299350de3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:33.084060 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-917e87a5-390d-4744-b5a2-f36299350de3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:33.096878 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:52:33.534054 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-917e87a5-390d-4744-b5a2-f36299350de3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:33.573150 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-917e87a5-390d-4744-b5a2-f36299350de3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.519315 Oct 24 08:52:33.574935 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 131/250] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:33 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:34.604315 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7f4ee938-b3f3-4a61-b2d7-023c75f41cb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:34.606446 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7f4ee938-b3f3-4a61-b2d7-023c75f41cb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:34.619484 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f4ee938-b3f3-4a61-b2d7-023c75f41cb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:34.620978 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f4ee938-b3f3-4a61-b2d7-023c75f41cb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:34.622222 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f4ee938-b3f3-4a61-b2d7-023c75f41cb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:34.624646 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f4ee938-b3f3-4a61-b2d7-023c75f41cb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:34.625374 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f4ee938-b3f3-4a61-b2d7-023c75f41cb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:34.626273 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7f4ee938-b3f3-4a61-b2d7-023c75f41cb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:34.922293 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7f4ee938-b3f3-4a61-b2d7-023c75f41cb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:34.936772 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7f4ee938-b3f3-4a61-b2d7-023c75f41cb3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.341083 Oct 24 08:52:34.937934 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 120/251] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:34 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:35.967497 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-448159d4-78b9-411d-b8da-83b0de872da1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:35.974763 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-448159d4-78b9-411d-b8da-83b0de872da1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:35.996598 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-448159d4-78b9-411d-b8da-83b0de872da1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:35.997248 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-448159d4-78b9-411d-b8da-83b0de872da1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:35.998490 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-448159d4-78b9-411d-b8da-83b0de872da1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:35.999816 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-448159d4-78b9-411d-b8da-83b0de872da1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:36.000724 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-448159d4-78b9-411d-b8da-83b0de872da1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:36.001567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-448159d4-78b9-411d-b8da-83b0de872da1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:36.275618 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-448159d4-78b9-411d-b8da-83b0de872da1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:36.289866 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-448159d4-78b9-411d-b8da-83b0de872da1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.331936 Oct 24 08:52:36.290895 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 132/252] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:35 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:37.318746 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c90286a7-8a33-4683-b72d-ea3472be5f7f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:37.320152 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c90286a7-8a33-4683-b72d-ea3472be5f7f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:37.332646 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c90286a7-8a33-4683-b72d-ea3472be5f7f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:37.335422 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c90286a7-8a33-4683-b72d-ea3472be5f7f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:37.336188 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c90286a7-8a33-4683-b72d-ea3472be5f7f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:37.337515 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c90286a7-8a33-4683-b72d-ea3472be5f7f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:37.338236 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c90286a7-8a33-4683-b72d-ea3472be5f7f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:37.339188 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c90286a7-8a33-4683-b72d-ea3472be5f7f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:37.531877 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c90286a7-8a33-4683-b72d-ea3472be5f7f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:37.544332 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c90286a7-8a33-4683-b72d-ea3472be5f7f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.233640 Oct 24 08:52:37.556883 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 121/253] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:37 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:38.569583 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-6e44446c-92d1-4718-aaba-6f8ac3f03079 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:38.570371 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-6e44446c-92d1-4718-aaba-6f8ac3f03079 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:38.578350 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6e44446c-92d1-4718-aaba-6f8ac3f03079 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:38.579132 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6e44446c-92d1-4718-aaba-6f8ac3f03079 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:38.579583 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6e44446c-92d1-4718-aaba-6f8ac3f03079 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:38.580364 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6e44446c-92d1-4718-aaba-6f8ac3f03079 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:38.584378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6e44446c-92d1-4718-aaba-6f8ac3f03079 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:38.584888 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6e44446c-92d1-4718-aaba-6f8ac3f03079 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:38.856036 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-6e44446c-92d1-4718-aaba-6f8ac3f03079 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:38.866393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-6e44446c-92d1-4718-aaba-6f8ac3f03079 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.300775 Oct 24 08:52:38.866979 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 133/254] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:38 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:52:39.900882 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-de0cf485-b39a-442f-bd95-0a4ba193746f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:39.901553 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-de0cf485-b39a-442f-bd95-0a4ba193746f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:39.908769 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-de0cf485-b39a-442f-bd95-0a4ba193746f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:39.909266 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-de0cf485-b39a-442f-bd95-0a4ba193746f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:39.909709 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-de0cf485-b39a-442f-bd95-0a4ba193746f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:39.910424 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-de0cf485-b39a-442f-bd95-0a4ba193746f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:39.910876 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-de0cf485-b39a-442f-bd95-0a4ba193746f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:39.911308 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-de0cf485-b39a-442f-bd95-0a4ba193746f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:40.209852 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-de0cf485-b39a-442f-bd95-0a4ba193746f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:40.220150 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-de0cf485-b39a-442f-bd95-0a4ba193746f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.322963 Oct 24 08:52:40.220832 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 122/255] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:39 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:41.245282 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-45ab49fb-9192-484a-8c83-aaf045b2dec0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:41.245904 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-45ab49fb-9192-484a-8c83-aaf045b2dec0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:41.253080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-45ab49fb-9192-484a-8c83-aaf045b2dec0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:41.253832 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-45ab49fb-9192-484a-8c83-aaf045b2dec0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:41.254444 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-45ab49fb-9192-484a-8c83-aaf045b2dec0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:41.255445 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-45ab49fb-9192-484a-8c83-aaf045b2dec0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:41.256064 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-45ab49fb-9192-484a-8c83-aaf045b2dec0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:41.256703 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-45ab49fb-9192-484a-8c83-aaf045b2dec0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:41.509598 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-45ab49fb-9192-484a-8c83-aaf045b2dec0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:41.519170 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-45ab49fb-9192-484a-8c83-aaf045b2dec0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1522 microversion: 2.1 time: 0.278859 Oct 24 08:52:41.519992 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 134/256] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:41 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1522 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:42.544437 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7babbf4c-5c65-4c33-ae04-3f65042b3dde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:42.545840 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7babbf4c-5c65-4c33-ae04-3f65042b3dde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:42.559349 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7babbf4c-5c65-4c33-ae04-3f65042b3dde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:42.562219 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7babbf4c-5c65-4c33-ae04-3f65042b3dde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:42.563407 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7babbf4c-5c65-4c33-ae04-3f65042b3dde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:42.564981 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7babbf4c-5c65-4c33-ae04-3f65042b3dde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:42.566155 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7babbf4c-5c65-4c33-ae04-3f65042b3dde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:42.568858 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7babbf4c-5c65-4c33-ae04-3f65042b3dde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:42.934418 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7babbf4c-5c65-4c33-ae04-3f65042b3dde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:42.951979 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7babbf4c-5c65-4c33-ae04-3f65042b3dde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1502 microversion: 2.1 time: 0.411748 Oct 24 08:52:42.957413 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 123/257] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:42 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1502 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:42.999376 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1e5b5821-3c8d-4010-b781-9b18c3482d7e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:43.006501 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1e5b5821-3c8d-4010-b781-9b18c3482d7e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:43.024777 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1e5b5821-3c8d-4010-b781-9b18c3482d7e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:43.025285 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1e5b5821-3c8d-4010-b781-9b18c3482d7e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:43.034973 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1e5b5821-3c8d-4010-b781-9b18c3482d7e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:43.041908 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1e5b5821-3c8d-4010-b781-9b18c3482d7e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:43.043494 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1e5b5821-3c8d-4010-b781-9b18c3482d7e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:43.045895 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1e5b5821-3c8d-4010-b781-9b18c3482d7e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:43.323565 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1e5b5821-3c8d-4010-b781-9b18c3482d7e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:43.352933 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1e5b5821-3c8d-4010-b781-9b18c3482d7e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1516 microversion: 2.1 time: 0.376536 Oct 24 08:52:43.356229 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 135/258] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:42 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1516 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:43.708213 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-d8f5247e-5517-4875-a288-25a05325b70e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Action: 'create', calling method: .version_select at 0x7fdfedfd6f70>, body: {"keypair": {"name": "tempest-keypair-1643738377", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBItSuTduPEARhkC8WKthZ0CLtF9cZY9lAjUUlP89d7U+zr/tJXYHPH8hwx0ixURebciNj05EDd/nF3T7yU61nIyDENlXfMJTOBvXWHLZXLiIMPsFl19Q2HPgcAyTTdjSAA=="}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:52:43.728715 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-d8f5247e-5517-4875-a288-25a05325b70e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Getting quotas for project 45be9d45ba4c4a2ca1ed053a7263dbee. Resources: {'key_pairs'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:52:43.737152 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-d8f5247e-5517-4875-a288-25a05325b70e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Getting quotas for user 94169e17266b4e6fad7e068a4c1384c7 and project 45be9d45ba4c4a2ca1ed053a7263dbee. Resources: {'key_pairs'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:52:43.789677 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-d8f5247e-5517-4875-a288-25a05325b70e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.233015 Oct 24 08:52:43.790614 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 124/259] 10.222.0.39 () {60 vars in 1159 bytes} [Mon Oct 24 08:52:43 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:52:45.269775 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1316053162", "imageRef": "bba331ba-faf5-4a56-81d9-6e84e9609665", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d5cd9b46-2491-4f10-bda2-40f7d635b974"}], "security_groups": [{"name": "tempest-securitygroup--1226777387"}], "key_name": "tempest-keypair-1643738377", "user_data": "}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:52:45.474110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.network.neutron [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] validate_networks() for [('d5cd9b46-2491-4f10-bda2-40f7d635b974', None, None, None, None, None)] {{(pid=130395) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 08:52:45.901262 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Flavor limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 08:52:45.901687 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Image limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 08:52:45.902158 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Flavor pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 08:52:45.902559 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Image pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 08:52:45.903751 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 08:52:45.913726 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-cec1e7bf-6f3e-40f8-85d2-e7193f12889c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:52:45.916390 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-cec1e7bf-6f3e-40f8-85d2-e7193f12889c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:45.922920 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:45.923795 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:45.924643 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:45.925965 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:45.926696 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cec1e7bf-6f3e-40f8-85d2-e7193f12889c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:45.927198 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cec1e7bf-6f3e-40f8-85d2-e7193f12889c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:45.927495 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:45.927672 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cec1e7bf-6f3e-40f8-85d2-e7193f12889c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:45.928510 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cec1e7bf-6f3e-40f8-85d2-e7193f12889c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:45.929185 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cec1e7bf-6f3e-40f8-85d2-e7193f12889c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:45.929330 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:45.929715 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cec1e7bf-6f3e-40f8-85d2-e7193f12889c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:45.954562 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Getting quotas for project 45be9d45ba4c4a2ca1ed053a7263dbee. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:52:45.957184 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.objects.instance [None req-cec1e7bf-6f3e-40f8-85d2-e7193f12889c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lazy-loading 'flavor' on Instance uuid e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 {{(pid=130396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:52:45.960955 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Getting quotas for user 94169e17266b4e6fad7e068a4c1384c7 and project 45be9d45ba4c4a2ca1ed053a7263dbee. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:52:45.975524 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Going to run 1 instances... {{(pid=130395) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 08:52:45.978002 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bba331ba-faf5-4a56-81d9-6e84e9609665',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130395) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 08:52:46.031277 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-31727068-922b-4cc8-ab05-0b1e0c152998 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.766630 Oct 24 08:52:46.032256 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 136/260] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 08:52:45 2022] POST /compute/v2.1/servers => generated 402 bytes in 769 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 24 08:52:46.047222 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-11233d08-f320-4821-bd56-79e88b7d8a52 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:46.048385 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-11233d08-f320-4821-bd56-79e88b7d8a52 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:46.102938 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-cec1e7bf-6f3e-40f8-85d2-e7193f12889c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "POST /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451/action" status: 200 len: 32 microversion: 2.1 time: 0.194003 Oct 24 08:52:46.109826 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 125/261] 10.222.0.39 () {60 vars in 1278 bytes} [Mon Oct 24 08:52:45 2022] POST /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451/action => generated 32 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 24 08:52:46.116344 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9423ee55-1d43-4045-bcbc-70ea192c1367 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:46.117053 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9423ee55-1d43-4045-bcbc-70ea192c1367 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:46.125761 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9423ee55-1d43-4045-bcbc-70ea192c1367 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:46.127275 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9423ee55-1d43-4045-bcbc-70ea192c1367 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:46.128081 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9423ee55-1d43-4045-bcbc-70ea192c1367 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:46.129036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9423ee55-1d43-4045-bcbc-70ea192c1367 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:46.129742 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9423ee55-1d43-4045-bcbc-70ea192c1367 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:46.130342 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9423ee55-1d43-4045-bcbc-70ea192c1367 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:46.131271 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-11233d08-f320-4821-bd56-79e88b7d8a52 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:46.142673 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-11233d08-f320-4821-bd56-79e88b7d8a52 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1238 microversion: 2.1 time: 0.101582 Oct 24 08:52:46.143797 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 137/262] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:46 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1238 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:46.291898 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9423ee55-1d43-4045-bcbc-70ea192c1367 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:46.300326 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9423ee55-1d43-4045-bcbc-70ea192c1367 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.188847 Oct 24 08:52:46.300828 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 126/263] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:46 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:47.167958 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-827cd7c3-dfdd-48b7-b3a3-19673810e558 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:47.169828 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-827cd7c3-dfdd-48b7-b3a3-19673810e558 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:47.177928 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-827cd7c3-dfdd-48b7-b3a3-19673810e558 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:47.178864 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-827cd7c3-dfdd-48b7-b3a3-19673810e558 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:47.179611 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-827cd7c3-dfdd-48b7-b3a3-19673810e558 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:47.180775 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-827cd7c3-dfdd-48b7-b3a3-19673810e558 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:47.181436 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-827cd7c3-dfdd-48b7-b3a3-19673810e558 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:47.182108 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-827cd7c3-dfdd-48b7-b3a3-19673810e558 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:47.262707 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-827cd7c3-dfdd-48b7-b3a3-19673810e558 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:47.271606 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-827cd7c3-dfdd-48b7-b3a3-19673810e558 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1290 microversion: 2.1 time: 0.107489 Oct 24 08:52:47.272821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 138/264] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:47 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1290 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:47.323863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-20eb91ce-1923-4b00-8818-3e5345fce7f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:47.328008 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-20eb91ce-1923-4b00-8818-3e5345fce7f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:47.337197 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-20eb91ce-1923-4b00-8818-3e5345fce7f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:47.338042 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-20eb91ce-1923-4b00-8818-3e5345fce7f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:47.338698 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-20eb91ce-1923-4b00-8818-3e5345fce7f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:47.339591 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-20eb91ce-1923-4b00-8818-3e5345fce7f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:47.340255 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-20eb91ce-1923-4b00-8818-3e5345fce7f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:47.341039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-20eb91ce-1923-4b00-8818-3e5345fce7f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:47.511485 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-20eb91ce-1923-4b00-8818-3e5345fce7f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:47.520748 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-20eb91ce-1923-4b00-8818-3e5345fce7f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.201408 Oct 24 08:52:47.521333 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 127/265] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:47 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:48.296848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-388b3c31-2967-41b7-ad80-ded24efa9d6b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:48.298144 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-388b3c31-2967-41b7-ad80-ded24efa9d6b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:48.299216 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=70,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:52:48.317788 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-388b3c31-2967-41b7-ad80-ded24efa9d6b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:48.318284 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-388b3c31-2967-41b7-ad80-ded24efa9d6b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:48.318725 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-388b3c31-2967-41b7-ad80-ded24efa9d6b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:48.319443 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-388b3c31-2967-41b7-ad80-ded24efa9d6b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:48.319893 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-388b3c31-2967-41b7-ad80-ded24efa9d6b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:48.320347 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-388b3c31-2967-41b7-ad80-ded24efa9d6b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:48.426277 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-388b3c31-2967-41b7-ad80-ded24efa9d6b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:48.434439 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-388b3c31-2967-41b7-ad80-ded24efa9d6b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1296 microversion: 2.1 time: 0.142953 Oct 24 08:52:48.435992 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 139/266] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:48 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1296 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:48.545362 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-03335bdf-dd0f-4f2d-9de3-31ec91ca71df tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:48.545978 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-03335bdf-dd0f-4f2d-9de3-31ec91ca71df tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:48.552988 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-03335bdf-dd0f-4f2d-9de3-31ec91ca71df tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:48.553470 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-03335bdf-dd0f-4f2d-9de3-31ec91ca71df tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:48.553902 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-03335bdf-dd0f-4f2d-9de3-31ec91ca71df tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:48.554605 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-03335bdf-dd0f-4f2d-9de3-31ec91ca71df tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:48.555057 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-03335bdf-dd0f-4f2d-9de3-31ec91ca71df tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:48.555478 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-03335bdf-dd0f-4f2d-9de3-31ec91ca71df tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:48.713302 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-03335bdf-dd0f-4f2d-9de3-31ec91ca71df tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:48.722215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-03335bdf-dd0f-4f2d-9de3-31ec91ca71df tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.181955 Oct 24 08:52:48.722800 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 128/267] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:48 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:49.460840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-cd37cd9c-9d3f-4d86-8275-757bc92f9ea3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:49.461475 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-cd37cd9c-9d3f-4d86-8275-757bc92f9ea3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:49.470426 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cd37cd9c-9d3f-4d86-8275-757bc92f9ea3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:49.470907 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cd37cd9c-9d3f-4d86-8275-757bc92f9ea3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:49.471342 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cd37cd9c-9d3f-4d86-8275-757bc92f9ea3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:49.472036 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cd37cd9c-9d3f-4d86-8275-757bc92f9ea3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:49.473100 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cd37cd9c-9d3f-4d86-8275-757bc92f9ea3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:49.473525 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cd37cd9c-9d3f-4d86-8275-757bc92f9ea3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:49.651284 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cd37cd9c-9d3f-4d86-8275-757bc92f9ea3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:49.660619 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-cd37cd9c-9d3f-4d86-8275-757bc92f9ea3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1364 microversion: 2.1 time: 0.204055 Oct 24 08:52:49.661336 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 140/268] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:49 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1364 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:49.719181 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-c10f9487-96b0-4dfb-a8bb-07e492967662 req-858f6fb5-4cae-49dd-b4c1-7ec1afa2ddd9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "89a2295a-90e9-4fc1-be3b-a130f7e71528", "tag": "47ad5200-c751-4a26-a72e-3a49c982245c"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:52:49.728080 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=61,nova_api:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:52:49.734326 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-c10f9487-96b0-4dfb-a8bb-07e492967662 req-858f6fb5-4cae-49dd-b4c1-7ec1afa2ddd9 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:49.734803 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-c10f9487-96b0-4dfb-a8bb-07e492967662 req-858f6fb5-4cae-49dd-b4c1-7ec1afa2ddd9 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:49.735233 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-c10f9487-96b0-4dfb-a8bb-07e492967662 req-858f6fb5-4cae-49dd-b4c1-7ec1afa2ddd9 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:49.751363 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e76d7883-d47d-4c5b-959d-0cb0146e211c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:49.752193 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-c10f9487-96b0-4dfb-a8bb-07e492967662 req-858f6fb5-4cae-49dd-b4c1-7ec1afa2ddd9 service nova] Creating event network-changed:47ad5200-c751-4a26-a72e-3a49c982245c for instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 on hv-cinder-maste Oct 24 08:52:49.752344 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e76d7883-d47d-4c5b-959d-0cb0146e211c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:49.757764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-c10f9487-96b0-4dfb-a8bb-07e492967662 req-858f6fb5-4cae-49dd-b4c1-7ec1afa2ddd9 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042264 Oct 24 08:52:49.758559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 129/269] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:52:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:52:49.759026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e76d7883-d47d-4c5b-959d-0cb0146e211c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:49.759610 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e76d7883-d47d-4c5b-959d-0cb0146e211c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:49.760170 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e76d7883-d47d-4c5b-959d-0cb0146e211c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:49.761024 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e76d7883-d47d-4c5b-959d-0cb0146e211c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:49.761485 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e76d7883-d47d-4c5b-959d-0cb0146e211c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:49.761830 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e76d7883-d47d-4c5b-959d-0cb0146e211c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:49.896375 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e76d7883-d47d-4c5b-959d-0cb0146e211c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:49.904303 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e76d7883-d47d-4c5b-959d-0cb0146e211c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.159162 Oct 24 08:52:49.905061 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 141/270] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:49 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:50.684373 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-337178e0-4827-4c70-95b8-c8b07d5568d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:50.685074 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-337178e0-4827-4c70-95b8-c8b07d5568d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:50.691220 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-337178e0-4827-4c70-95b8-c8b07d5568d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:50.691702 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-337178e0-4827-4c70-95b8-c8b07d5568d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:50.692155 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-337178e0-4827-4c70-95b8-c8b07d5568d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:50.692848 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-337178e0-4827-4c70-95b8-c8b07d5568d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:50.693302 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-337178e0-4827-4c70-95b8-c8b07d5568d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:50.693723 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-337178e0-4827-4c70-95b8-c8b07d5568d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:50.835787 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-337178e0-4827-4c70-95b8-c8b07d5568d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:50.845345 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-337178e0-4827-4c70-95b8-c8b07d5568d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1364 microversion: 2.1 time: 0.165236 Oct 24 08:52:50.845915 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 130/271] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:50 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1364 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:50.931189 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7ec8cf7d-7626-422a-8ec2-8447dee4e095 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:50.931792 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7ec8cf7d-7626-422a-8ec2-8447dee4e095 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:50.938351 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7ec8cf7d-7626-422a-8ec2-8447dee4e095 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:50.938831 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7ec8cf7d-7626-422a-8ec2-8447dee4e095 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:50.939266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7ec8cf7d-7626-422a-8ec2-8447dee4e095 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:50.939952 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7ec8cf7d-7626-422a-8ec2-8447dee4e095 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:50.940434 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7ec8cf7d-7626-422a-8ec2-8447dee4e095 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:50.940858 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7ec8cf7d-7626-422a-8ec2-8447dee4e095 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:51.072639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7ec8cf7d-7626-422a-8ec2-8447dee4e095 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:51.082157 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7ec8cf7d-7626-422a-8ec2-8447dee4e095 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.154798 Oct 24 08:52:51.082716 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 142/272] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:50 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:51.868935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-32a3505c-8631-42a2-bd86-63e8cb598d41 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:51.869532 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-32a3505c-8631-42a2-bd86-63e8cb598d41 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:51.877287 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32a3505c-8631-42a2-bd86-63e8cb598d41 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:51.877813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32a3505c-8631-42a2-bd86-63e8cb598d41 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:51.878314 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32a3505c-8631-42a2-bd86-63e8cb598d41 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:51.879028 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32a3505c-8631-42a2-bd86-63e8cb598d41 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:51.879544 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32a3505c-8631-42a2-bd86-63e8cb598d41 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:51.880003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32a3505c-8631-42a2-bd86-63e8cb598d41 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:52.035453 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-32a3505c-8631-42a2-bd86-63e8cb598d41 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:52.044396 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-32a3505c-8631-42a2-bd86-63e8cb598d41 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1364 microversion: 2.1 time: 0.179280 Oct 24 08:52:52.045384 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 131/273] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:51 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1364 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:52.105745 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-9c66f1be-1865-492c-bf0f-47bb9b1e2dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:52.107362 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-9c66f1be-1865-492c-bf0f-47bb9b1e2dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:52.113294 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c66f1be-1865-492c-bf0f-47bb9b1e2dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:52.113766 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c66f1be-1865-492c-bf0f-47bb9b1e2dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:52.114195 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c66f1be-1865-492c-bf0f-47bb9b1e2dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:52.114877 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c66f1be-1865-492c-bf0f-47bb9b1e2dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:52.115589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c66f1be-1865-492c-bf0f-47bb9b1e2dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:52.116005 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c66f1be-1865-492c-bf0f-47bb9b1e2dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:52.252771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-9c66f1be-1865-492c-bf0f-47bb9b1e2dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:52.261932 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-9c66f1be-1865-492c-bf0f-47bb9b1e2dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.160029 Oct 24 08:52:52.262890 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 143/274] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:52 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:53.071214 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-36f09129-2c00-4412-9e50-fdd2847728c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:53.071982 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-36f09129-2c00-4412-9e50-fdd2847728c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:53.079144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-36f09129-2c00-4412-9e50-fdd2847728c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:53.079625 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-36f09129-2c00-4412-9e50-fdd2847728c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:53.080055 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-36f09129-2c00-4412-9e50-fdd2847728c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:53.080773 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-36f09129-2c00-4412-9e50-fdd2847728c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:53.081220 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-36f09129-2c00-4412-9e50-fdd2847728c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:53.081638 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-36f09129-2c00-4412-9e50-fdd2847728c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:53.226366 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-36f09129-2c00-4412-9e50-fdd2847728c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:53.235224 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-36f09129-2c00-4412-9e50-fdd2847728c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1364 microversion: 2.1 time: 0.168270 Oct 24 08:52:53.235852 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 132/275] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:53 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1364 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:53.285389 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-5e8ce06b-f015-4078-be01-5ba930c77d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:53.286004 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-5e8ce06b-f015-4078-be01-5ba930c77d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:53.292100 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e8ce06b-f015-4078-be01-5ba930c77d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:53.292608 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e8ce06b-f015-4078-be01-5ba930c77d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:53.293046 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e8ce06b-f015-4078-be01-5ba930c77d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:53.293737 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e8ce06b-f015-4078-be01-5ba930c77d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:53.294188 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e8ce06b-f015-4078-be01-5ba930c77d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:53.294606 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e8ce06b-f015-4078-be01-5ba930c77d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:53.468966 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-5e8ce06b-f015-4078-be01-5ba930c77d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:53.480667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-5e8ce06b-f015-4078-be01-5ba930c77d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.198899 Oct 24 08:52:53.482661 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 144/276] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:53 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:54.259011 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5ae54b06-e2d5-487e-97aa-da8ab525204d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:54.259698 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-5ae54b06-e2d5-487e-97aa-da8ab525204d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:54.261760 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=4 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:52:54.268001 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5ae54b06-e2d5-487e-97aa-da8ab525204d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:54.268671 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5ae54b06-e2d5-487e-97aa-da8ab525204d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:54.269196 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5ae54b06-e2d5-487e-97aa-da8ab525204d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:54.269982 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5ae54b06-e2d5-487e-97aa-da8ab525204d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:54.270504 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5ae54b06-e2d5-487e-97aa-da8ab525204d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:54.272973 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5ae54b06-e2d5-487e-97aa-da8ab525204d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:54.428798 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5ae54b06-e2d5-487e-97aa-da8ab525204d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:54.437814 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5ae54b06-e2d5-487e-97aa-da8ab525204d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1364 microversion: 2.1 time: 0.182561 Oct 24 08:52:54.438590 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 133/277] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:54 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1364 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:54.505835 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a2a643a5-9401-4da2-9218-26d8d886b344 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:54.506531 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a2a643a5-9401-4da2-9218-26d8d886b344 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:54.513239 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a2a643a5-9401-4da2-9218-26d8d886b344 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:54.513657 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a2a643a5-9401-4da2-9218-26d8d886b344 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:54.514079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a2a643a5-9401-4da2-9218-26d8d886b344 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:54.514857 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a2a643a5-9401-4da2-9218-26d8d886b344 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:54.515329 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a2a643a5-9401-4da2-9218-26d8d886b344 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:54.515750 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a2a643a5-9401-4da2-9218-26d8d886b344 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:54.670264 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a2a643a5-9401-4da2-9218-26d8d886b344 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:54.681305 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a2a643a5-9401-4da2-9218-26d8d886b344 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.178729 Oct 24 08:52:54.694015 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 145/278] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:54 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:55.468587 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-026ac406-3d36-4c8c-b098-c75ceeb344ca tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:55.469910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-026ac406-3d36-4c8c-b098-c75ceeb344ca tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:55.479833 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-026ac406-3d36-4c8c-b098-c75ceeb344ca tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:55.480710 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-026ac406-3d36-4c8c-b098-c75ceeb344ca tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:55.482289 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-026ac406-3d36-4c8c-b098-c75ceeb344ca tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:55.483286 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-026ac406-3d36-4c8c-b098-c75ceeb344ca tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:55.484044 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-026ac406-3d36-4c8c-b098-c75ceeb344ca tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:55.485083 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-026ac406-3d36-4c8c-b098-c75ceeb344ca tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:55.709258 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7c961763-b1ad-4958-9b39-e053c2b1fdfb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:55.710533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7c961763-b1ad-4958-9b39-e053c2b1fdfb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:55.719086 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7c961763-b1ad-4958-9b39-e053c2b1fdfb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:55.719681 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7c961763-b1ad-4958-9b39-e053c2b1fdfb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:55.720156 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7c961763-b1ad-4958-9b39-e053c2b1fdfb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:55.720924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7c961763-b1ad-4958-9b39-e053c2b1fdfb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:55.721387 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7c961763-b1ad-4958-9b39-e053c2b1fdfb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:55.721822 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7c961763-b1ad-4958-9b39-e053c2b1fdfb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:55.932877 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-026ac406-3d36-4c8c-b098-c75ceeb344ca tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:55.947791 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7c961763-b1ad-4958-9b39-e053c2b1fdfb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:55.948910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-026ac406-3d36-4c8c-b098-c75ceeb344ca tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1364 microversion: 2.1 time: 0.490045 Oct 24 08:52:55.950427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 134/279] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:55 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1364 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:55.964307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=2 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:52:55.987494 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7c961763-b1ad-4958-9b39-e053c2b1fdfb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.281985 Oct 24 08:52:55.988424 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 146/280] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:55 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:56.911568 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-5a8b325b-34b3-4844-90d9-c159feb2843b req-ea486cdf-c942-4839-b53f-9b3c80666627 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "89a2295a-90e9-4fc1-be3b-a130f7e71528", "name": "network-vif-plugged", "status": "completed", "tag": "47ad5200-c751-4a26-a72e-3a49c982245c"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:52:56.923214 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-5a8b325b-34b3-4844-90d9-c159feb2843b req-ea486cdf-c942-4839-b53f-9b3c80666627 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:56.923877 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-5a8b325b-34b3-4844-90d9-c159feb2843b req-ea486cdf-c942-4839-b53f-9b3c80666627 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:56.924357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-5a8b325b-34b3-4844-90d9-c159feb2843b req-ea486cdf-c942-4839-b53f-9b3c80666627 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:56.937612 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-5a8b325b-34b3-4844-90d9-c159feb2843b req-ea486cdf-c942-4839-b53f-9b3c80666627 service nova] Creating event network-vif-plugged:47ad5200-c751-4a26-a72e-3a49c982245c for instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 on hv-cinder-maste Oct 24 08:52:56.943954 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-5a8b325b-34b3-4844-90d9-c159feb2843b req-ea486cdf-c942-4839-b53f-9b3c80666627 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036207 Oct 24 08:52:56.945636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 135/281] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:52:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:52:56.993137 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-05fea756-dc57-404b-b48f-82607ce04b70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:56.993770 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-05fea756-dc57-404b-b48f-82607ce04b70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:56.995694 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=8 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:52:57.002064 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-05fea756-dc57-404b-b48f-82607ce04b70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:57.002552 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-05fea756-dc57-404b-b48f-82607ce04b70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:57.002997 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-05fea756-dc57-404b-b48f-82607ce04b70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:57.003712 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-05fea756-dc57-404b-b48f-82607ce04b70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:57.004183 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-05fea756-dc57-404b-b48f-82607ce04b70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:57.004606 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-05fea756-dc57-404b-b48f-82607ce04b70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:57.028404 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9d285afe-e863-491f-93e3-bfe9e08920d9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:57.029274 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9d285afe-e863-491f-93e3-bfe9e08920d9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:57.036214 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9d285afe-e863-491f-93e3-bfe9e08920d9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:57.036714 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9d285afe-e863-491f-93e3-bfe9e08920d9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:57.037158 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9d285afe-e863-491f-93e3-bfe9e08920d9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:57.037931 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9d285afe-e863-491f-93e3-bfe9e08920d9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:57.038391 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9d285afe-e863-491f-93e3-bfe9e08920d9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:57.038820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9d285afe-e863-491f-93e3-bfe9e08920d9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:57.043601 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:52:57.230870 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-05fea756-dc57-404b-b48f-82607ce04b70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:57.243341 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-05fea756-dc57-404b-b48f-82607ce04b70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1364 microversion: 2.1 time: 0.260172 Oct 24 08:52:57.244407 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 147/282] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:56 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1364 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:57.253737 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9d285afe-e863-491f-93e3-bfe9e08920d9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:57.262285 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9d285afe-e863-491f-93e3-bfe9e08920d9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.238869 Oct 24 08:52:57.262953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 136/283] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:57 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:58.267606 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-737b525b-faf4-481e-bc76-e6650d9f7167 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:58.268210 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-737b525b-faf4-481e-bc76-e6650d9f7167 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:58.274193 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-737b525b-faf4-481e-bc76-e6650d9f7167 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:58.274687 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-737b525b-faf4-481e-bc76-e6650d9f7167 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:58.275135 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-737b525b-faf4-481e-bc76-e6650d9f7167 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:58.275848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-737b525b-faf4-481e-bc76-e6650d9f7167 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:58.276338 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-737b525b-faf4-481e-bc76-e6650d9f7167 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:58.276772 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-737b525b-faf4-481e-bc76-e6650d9f7167 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:58.301914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8ef6ea94-20e8-44cb-801b-94ee20d956d1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:58.302890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8ef6ea94-20e8-44cb-801b-94ee20d956d1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:58.312751 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8ef6ea94-20e8-44cb-801b-94ee20d956d1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:58.313249 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8ef6ea94-20e8-44cb-801b-94ee20d956d1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:58.313806 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8ef6ea94-20e8-44cb-801b-94ee20d956d1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:58.314837 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8ef6ea94-20e8-44cb-801b-94ee20d956d1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:58.315433 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8ef6ea94-20e8-44cb-801b-94ee20d956d1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:58.316061 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8ef6ea94-20e8-44cb-801b-94ee20d956d1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:58.479701 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-737b525b-faf4-481e-bc76-e6650d9f7167 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:58.491199 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-737b525b-faf4-481e-bc76-e6650d9f7167 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1364 microversion: 2.1 time: 0.227545 Oct 24 08:52:58.492377 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 148/284] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:58 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1364 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:58.518736 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8ef6ea94-20e8-44cb-801b-94ee20d956d1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:58.529256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8ef6ea94-20e8-44cb-801b-94ee20d956d1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.233722 Oct 24 08:52:58.530156 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 137/285] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:58 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:59.516347 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-9f65a717-558d-4248-a728-657feed8e3e1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:59.517044 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-9f65a717-558d-4248-a728-657feed8e3e1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:59.523170 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9f65a717-558d-4248-a728-657feed8e3e1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:59.523654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9f65a717-558d-4248-a728-657feed8e3e1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:59.524091 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9f65a717-558d-4248-a728-657feed8e3e1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:59.524838 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9f65a717-558d-4248-a728-657feed8e3e1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:59.525285 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9f65a717-558d-4248-a728-657feed8e3e1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:59.525709 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9f65a717-558d-4248-a728-657feed8e3e1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:59.542574 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=100 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:52:59.553785 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-1b812229-44c9-45e2-a74e-feca64592c71 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:52:59.554526 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-1b812229-44c9-45e2-a74e-feca64592c71 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:52:59.562635 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1b812229-44c9-45e2-a74e-feca64592c71 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:59.563622 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1b812229-44c9-45e2-a74e-feca64592c71 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:59.564450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1b812229-44c9-45e2-a74e-feca64592c71 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:59.565768 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1b812229-44c9-45e2-a74e-feca64592c71 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:52:59.566709 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1b812229-44c9-45e2-a74e-feca64592c71 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:52:59.567514 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1b812229-44c9-45e2-a74e-feca64592c71 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:52:59.764269 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-9f65a717-558d-4248-a728-657feed8e3e1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:59.772345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-9f65a717-558d-4248-a728-657feed8e3e1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1364 microversion: 2.1 time: 0.261071 Oct 24 08:52:59.772959 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 149/286] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:59 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1364 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:52:59.780547 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-1b812229-44c9-45e2-a74e-feca64592c71 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:52:59.788890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-1b812229-44c9-45e2-a74e-feca64592c71 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.240049 Oct 24 08:52:59.789483 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 138/287] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:52:59 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:00.800919 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-02250ecb-0080-4346-ac09-bc380d9739e7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:00.801529 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-02250ecb-0080-4346-ac09-bc380d9739e7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:00.809762 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02250ecb-0080-4346-ac09-bc380d9739e7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:00.810643 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02250ecb-0080-4346-ac09-bc380d9739e7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:00.811365 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02250ecb-0080-4346-ac09-bc380d9739e7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:00.812612 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02250ecb-0080-4346-ac09-bc380d9739e7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:00.813336 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02250ecb-0080-4346-ac09-bc380d9739e7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:00.814076 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02250ecb-0080-4346-ac09-bc380d9739e7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:00.816215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-1ec71f7a-07a4-44ff-99a2-317355f76c11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:00.817288 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-1ec71f7a-07a4-44ff-99a2-317355f76c11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:00.826337 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1ec71f7a-07a4-44ff-99a2-317355f76c11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:00.826833 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1ec71f7a-07a4-44ff-99a2-317355f76c11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:00.827279 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1ec71f7a-07a4-44ff-99a2-317355f76c11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:00.827993 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1ec71f7a-07a4-44ff-99a2-317355f76c11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:00.828477 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1ec71f7a-07a4-44ff-99a2-317355f76c11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:00.828907 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1ec71f7a-07a4-44ff-99a2-317355f76c11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:01.044350 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-1ec71f7a-07a4-44ff-99a2-317355f76c11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:01.048214 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-02250ecb-0080-4346-ac09-bc380d9739e7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:01.053827 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-1ec71f7a-07a4-44ff-99a2-317355f76c11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.244970 Oct 24 08:53:01.054417 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 139/288] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:00 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:01.056553 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-02250ecb-0080-4346-ac09-bc380d9739e7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1364 microversion: 2.1 time: 0.264040 Oct 24 08:53:01.057138 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 150/289] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:00 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1364 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:02.083676 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7be131ea-381d-4286-83db-b6f3ad32b393 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:02.083676 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7be131ea-381d-4286-83db-b6f3ad32b393 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:02.084653 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-4a2e804f-5157-4363-a472-691e1abfdaf3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:02.084653 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-4a2e804f-5157-4363-a472-691e1abfdaf3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:02.087037 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4a2e804f-5157-4363-a472-691e1abfdaf3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:02.087687 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7be131ea-381d-4286-83db-b6f3ad32b393 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:02.087969 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4a2e804f-5157-4363-a472-691e1abfdaf3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:02.088269 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7be131ea-381d-4286-83db-b6f3ad32b393 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:02.088432 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4a2e804f-5157-4363-a472-691e1abfdaf3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:02.088714 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7be131ea-381d-4286-83db-b6f3ad32b393 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:02.089150 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4a2e804f-5157-4363-a472-691e1abfdaf3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:02.089422 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7be131ea-381d-4286-83db-b6f3ad32b393 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:02.089650 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4a2e804f-5157-4363-a472-691e1abfdaf3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:02.089915 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7be131ea-381d-4286-83db-b6f3ad32b393 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:02.090069 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4a2e804f-5157-4363-a472-691e1abfdaf3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:02.090999 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7be131ea-381d-4286-83db-b6f3ad32b393 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:02.289440 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-4a2e804f-5157-4363-a472-691e1abfdaf3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:02.296446 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7be131ea-381d-4286-83db-b6f3ad32b393 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:02.297858 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-4a2e804f-5157-4363-a472-691e1abfdaf3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1364 microversion: 2.1 time: 0.222068 Oct 24 08:53:02.298473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 151/290] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:02 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1364 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:02.305212 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7be131ea-381d-4286-83db-b6f3ad32b393 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.230778 Oct 24 08:53:02.305987 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 140/291] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:02 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:03.320674 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a8fa8771-2af5-40ec-8a59-14ec89658abf tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:03.321274 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a8fa8771-2af5-40ec-8a59-14ec89658abf tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:03.327414 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a8fa8771-2af5-40ec-8a59-14ec89658abf tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:03.327904 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a8fa8771-2af5-40ec-8a59-14ec89658abf tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:03.328373 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a8fa8771-2af5-40ec-8a59-14ec89658abf tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:03.329002 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4cd179ca-3e14-478b-8103-268e9099c2f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:03.329409 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a8fa8771-2af5-40ec-8a59-14ec89658abf tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:03.329518 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a8fa8771-2af5-40ec-8a59-14ec89658abf tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:03.329617 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4cd179ca-3e14-478b-8103-268e9099c2f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:03.329924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a8fa8771-2af5-40ec-8a59-14ec89658abf tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:03.335662 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cd179ca-3e14-478b-8103-268e9099c2f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:03.336188 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cd179ca-3e14-478b-8103-268e9099c2f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:03.336635 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cd179ca-3e14-478b-8103-268e9099c2f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:03.337334 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cd179ca-3e14-478b-8103-268e9099c2f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:03.337821 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cd179ca-3e14-478b-8103-268e9099c2f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:03.338249 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cd179ca-3e14-478b-8103-268e9099c2f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:03.522965 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a8fa8771-2af5-40ec-8a59-14ec89658abf tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:03.534392 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a8fa8771-2af5-40ec-8a59-14ec89658abf tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1364 microversion: 2.1 time: 0.217478 Oct 24 08:53:03.534698 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4cd179ca-3e14-478b-8103-268e9099c2f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:03.535421 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 152/292] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:03 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1364 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:03.546446 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4cd179ca-3e14-478b-8103-268e9099c2f2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.221539 Oct 24 08:53:03.547309 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 141/293] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:03 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:04.557321 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-593500bf-07a7-43d2-acc7-ae30fd8e3a2e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:04.557830 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-593500bf-07a7-43d2-acc7-ae30fd8e3a2e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:04.564464 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-593500bf-07a7-43d2-acc7-ae30fd8e3a2e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:04.564944 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-593500bf-07a7-43d2-acc7-ae30fd8e3a2e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:04.565479 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-593500bf-07a7-43d2-acc7-ae30fd8e3a2e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:04.566138 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-593500bf-07a7-43d2-acc7-ae30fd8e3a2e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:04.566587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-593500bf-07a7-43d2-acc7-ae30fd8e3a2e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:04.567004 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-593500bf-07a7-43d2-acc7-ae30fd8e3a2e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:04.570624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c788b6cf-cc7e-4324-8438-cbbd4d37d0c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:04.571259 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c788b6cf-cc7e-4324-8438-cbbd4d37d0c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:04.577824 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c788b6cf-cc7e-4324-8438-cbbd4d37d0c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:04.578309 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c788b6cf-cc7e-4324-8438-cbbd4d37d0c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:04.578751 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c788b6cf-cc7e-4324-8438-cbbd4d37d0c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:04.579451 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c788b6cf-cc7e-4324-8438-cbbd4d37d0c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:04.579910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c788b6cf-cc7e-4324-8438-cbbd4d37d0c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:04.580366 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c788b6cf-cc7e-4324-8438-cbbd4d37d0c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:04.774226 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-593500bf-07a7-43d2-acc7-ae30fd8e3a2e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:04.782969 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-593500bf-07a7-43d2-acc7-ae30fd8e3a2e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1364 microversion: 2.1 time: 0.229499 Oct 24 08:53:04.783552 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 153/294] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:04 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1364 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:04.785833 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c788b6cf-cc7e-4324-8438-cbbd4d37d0c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:04.794237 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c788b6cf-cc7e-4324-8438-cbbd4d37d0c7 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.226684 Oct 24 08:53:04.795007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 142/295] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:04 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:05.816661 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ea122c79-48e5-431d-a4ee-10c4942e90ba tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:05.817323 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8576e942-2fc0-4043-a77b-1408734e907b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:05.817690 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ea122c79-48e5-431d-a4ee-10c4942e90ba tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:05.817790 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8576e942-2fc0-4043-a77b-1408734e907b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:05.823723 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ea122c79-48e5-431d-a4ee-10c4942e90ba tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:05.824247 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ea122c79-48e5-431d-a4ee-10c4942e90ba tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:05.825436 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8576e942-2fc0-4043-a77b-1408734e907b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:05.826078 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ea122c79-48e5-431d-a4ee-10c4942e90ba tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:05.826167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ea122c79-48e5-431d-a4ee-10c4942e90ba tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:05.826258 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8576e942-2fc0-4043-a77b-1408734e907b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:05.826634 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ea122c79-48e5-431d-a4ee-10c4942e90ba tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:05.826909 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8576e942-2fc0-4043-a77b-1408734e907b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:05.827091 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ea122c79-48e5-431d-a4ee-10c4942e90ba tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:05.829740 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8576e942-2fc0-4043-a77b-1408734e907b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:05.830219 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8576e942-2fc0-4043-a77b-1408734e907b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:05.830653 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8576e942-2fc0-4043-a77b-1408734e907b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:06.029121 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ea122c79-48e5-431d-a4ee-10c4942e90ba tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:06.031977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8576e942-2fc0-4043-a77b-1408734e907b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:06.037640 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ea122c79-48e5-431d-a4ee-10c4942e90ba tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1364 microversion: 2.1 time: 0.225062 Oct 24 08:53:06.038207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 154/296] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:05 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1364 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:06.039665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8576e942-2fc0-4043-a77b-1408734e907b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.226357 Oct 24 08:53:06.040325 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 143/297] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:05 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:07.062914 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-32b7e4bb-616f-4504-b369-61f00e4dc0ae tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:07.062914 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-32b7e4bb-616f-4504-b369-61f00e4dc0ae tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:07.064499 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-47e9fd12-bd85-4215-b60c-ac33f4e82b0d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:07.065689 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-47e9fd12-bd85-4215-b60c-ac33f4e82b0d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:07.069602 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-32b7e4bb-616f-4504-b369-61f00e4dc0ae tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:07.070130 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-32b7e4bb-616f-4504-b369-61f00e4dc0ae tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:07.070595 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-32b7e4bb-616f-4504-b369-61f00e4dc0ae tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:07.071365 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-32b7e4bb-616f-4504-b369-61f00e4dc0ae tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:07.071841 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-32b7e4bb-616f-4504-b369-61f00e4dc0ae tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:07.072153 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47e9fd12-bd85-4215-b60c-ac33f4e82b0d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:07.072369 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-32b7e4bb-616f-4504-b369-61f00e4dc0ae tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:07.072654 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47e9fd12-bd85-4215-b60c-ac33f4e82b0d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:07.073097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47e9fd12-bd85-4215-b60c-ac33f4e82b0d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:07.073817 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47e9fd12-bd85-4215-b60c-ac33f4e82b0d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:07.074271 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47e9fd12-bd85-4215-b60c-ac33f4e82b0d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:07.075464 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47e9fd12-bd85-4215-b60c-ac33f4e82b0d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:07.298469 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-32b7e4bb-616f-4504-b369-61f00e4dc0ae tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:07.304305 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-47e9fd12-bd85-4215-b60c-ac33f4e82b0d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:07.307123 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-32b7e4bb-616f-4504-b369-61f00e4dc0ae tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1364 microversion: 2.1 time: 0.249427 Oct 24 08:53:07.308251 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 155/298] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:07 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1364 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:07.313370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-47e9fd12-bd85-4215-b60c-ac33f4e82b0d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.253196 Oct 24 08:53:07.314060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 144/299] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:07 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:08.331531 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-05d2d006-2d59-473a-9ad9-11f970a4efa5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:08.332057 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-05d2d006-2d59-473a-9ad9-11f970a4efa5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:08.337167 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8e9d8e31-dd73-4876-98f6-9205504f3c4a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:08.337867 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8e9d8e31-dd73-4876-98f6-9205504f3c4a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:08.338029 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-05d2d006-2d59-473a-9ad9-11f970a4efa5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:08.338525 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-05d2d006-2d59-473a-9ad9-11f970a4efa5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:08.338963 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-05d2d006-2d59-473a-9ad9-11f970a4efa5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:08.339666 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-05d2d006-2d59-473a-9ad9-11f970a4efa5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:08.340668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-05d2d006-2d59-473a-9ad9-11f970a4efa5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:08.341102 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-05d2d006-2d59-473a-9ad9-11f970a4efa5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:08.348508 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8e9d8e31-dd73-4876-98f6-9205504f3c4a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:08.349015 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8e9d8e31-dd73-4876-98f6-9205504f3c4a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:08.349460 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8e9d8e31-dd73-4876-98f6-9205504f3c4a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:08.350178 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8e9d8e31-dd73-4876-98f6-9205504f3c4a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:08.350628 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8e9d8e31-dd73-4876-98f6-9205504f3c4a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:08.351044 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8e9d8e31-dd73-4876-98f6-9205504f3c4a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:08.559731 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-05d2d006-2d59-473a-9ad9-11f970a4efa5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:08.567749 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-05d2d006-2d59-473a-9ad9-11f970a4efa5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1549 microversion: 2.1 time: 0.240054 Oct 24 08:53:08.568377 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 156/300] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:08 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1549 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:08.570292 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8e9d8e31-dd73-4876-98f6-9205504f3c4a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:08.578798 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8e9d8e31-dd73-4876-98f6-9205504f3c4a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1502 microversion: 2.1 time: 0.245347 Oct 24 08:53:08.579364 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 145/301] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:08 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1502 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:08.591900 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8ce2c07d-2883-433a-8fc8-0db58da9badd tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:08.592512 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8ce2c07d-2883-433a-8fc8-0db58da9badd tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:08.598511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ce2c07d-2883-433a-8fc8-0db58da9badd tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:08.599001 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ce2c07d-2883-433a-8fc8-0db58da9badd tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:08.599436 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ce2c07d-2883-433a-8fc8-0db58da9badd tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:08.600159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ce2c07d-2883-433a-8fc8-0db58da9badd tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:08.600608 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ce2c07d-2883-433a-8fc8-0db58da9badd tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:08.601033 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ce2c07d-2883-433a-8fc8-0db58da9badd tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:08.602159 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6b07f7e2-fd14-4dec-864d-77ac38187e05 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6ffef37f-badb-4c2f-825a-f98b3ca9481d"}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:53:08.604534 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6b07f7e2-fd14-4dec-864d-77ac38187e05 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:08.611298 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b07f7e2-fd14-4dec-864d-77ac38187e05 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:08.611783 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b07f7e2-fd14-4dec-864d-77ac38187e05 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:08.612242 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b07f7e2-fd14-4dec-864d-77ac38187e05 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:08.612950 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b07f7e2-fd14-4dec-864d-77ac38187e05 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:08.613404 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b07f7e2-fd14-4dec-864d-77ac38187e05 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:08.613831 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b07f7e2-fd14-4dec-864d-77ac38187e05 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:08.637953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-6b07f7e2-fd14-4dec-864d-77ac38187e05 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] HTTP exception thrown: Cannot 'attach_volume' instance e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 while it is in vm_state rescued Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6b07f7e2-fd14-4dec-864d-77ac38187e05 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Request method failure captured: Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: POST /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451/os-volume_attachments HTTP/1.1 Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 74 Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlHvWm_27e2NE7FPo6k_ap-1fpqaBaXju_q7yx4ngEwubmyCghP7hRGkUjD3Kh0InrG1jri_Jf3QmFeB_Ar9fw71J3GQum71pn1j3lDf69J9HnlAUxEpDFaBqRZj8Qx4DvfjLqnUdxgNhetuy9MPxaavyu2rUfHlTx9TxAVKDfIab50 Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: fd4b77e78f9d4286aa92f6f90ff8b797 Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-583791902 Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 08:53:08.639650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 08:53:08.641004 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/fd4b77e78f9d4286aa92f6f90ff8b797"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_fd4b77e78f9d4286aa92f6f90ff8b797", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/fd4b77e78f9d4286aa92f6f90ff8b797"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/fd4b77e78f9d4286aa92f6f90ff8b797"}]}] Oct 24 08:53:08.641004 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-ServerRescueNegativeTestJSON-583791902 Oct 24 08:53:08.641004 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: fd4b77e78f9d4286aa92f6f90ff8b797 Oct 24 08:53:08.641004 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-583791902 Oct 24 08:53:08.641004 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-ServerRescueNegativeTestJSON-583791902-project Oct 24 08:53:08.641004 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 08:53:08.641004 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 08:53:08.641004 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 7127613bde784330bec364f02180291e Oct 24 08:53:08.641004 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-ServerRescueNegativeTestJSON-583791902-project Oct 24 08:53:08.641004 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Oct 24 08:53:08.641004 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {"volumeAttachment": {"volumeId": "6ffef37f-badb-4c2f-825a-f98b3ca9481d"}} Oct 24 08:53:08.641004 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Cannot 'attach_volume' instance e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 while it is in vm_state rescued Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'server_id': 'e7b4b4a1-6d10-4eef-94b3-395c2fc1c451', 'body': {'volumeAttachment': {'volumeId': '6ffef37f-badb-4c2f-825a-f98b3ca9481d'}}} Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 in vm_state rescued. Cannot attach_volume while the instance is in this state. Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:53:08.641764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 while it is in vm_state rescued Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:53:08.642935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:53:08.643976 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:53:08.643976 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:53:08.643976 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 while it is in vm_state rescued Oct 24 08:53:08.643976 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:53:08.643976 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6b07f7e2-fd14-4dec-864d-77ac38187e05 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Returning 409 to user: Cannot 'attach_volume' instance e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 while it is in vm_state rescued {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:53:08.643976 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6b07f7e2-fd14-4dec-864d-77ac38187e05 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "POST /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.042850 Oct 24 08:53:08.643976 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 146/302] 10.222.0.39 () {60 vars in 1323 bytes} [Mon Oct 24 08:53:08 2022] POST /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451/os-volume_attachments => generated 152 bytes in 44 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Oct 24 08:53:08.650056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-bc5703b8-e694-40ef-9604-452ea7c8f89c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:53:08.650591 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-bc5703b8-e694-40ef-9604-452ea7c8f89c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:08.657370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc5703b8-e694-40ef-9604-452ea7c8f89c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:08.657856 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc5703b8-e694-40ef-9604-452ea7c8f89c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:08.658296 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc5703b8-e694-40ef-9604-452ea7c8f89c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:08.658988 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc5703b8-e694-40ef-9604-452ea7c8f89c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:08.659442 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc5703b8-e694-40ef-9604-452ea7c8f89c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:08.659870 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc5703b8-e694-40ef-9604-452ea7c8f89c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:08.682443 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8ce2c07d-2883-433a-8fc8-0db58da9badd tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.095053 Oct 24 08:53:08.683213 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 157/303] 10.222.0.39 () {58 vars in 1275 bytes} [Mon Oct 24 08:53:08 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/os-interface => generated 284 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:53:08.729818 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-bc5703b8-e694-40ef-9604-452ea7c8f89c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "POST /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451/action" status: 202 len: 0 microversion: 2.1 time: 0.083174 Oct 24 08:53:08.734219 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 147/304] 10.222.0.39 () {60 vars in 1278 bytes} [Mon Oct 24 08:53:08 2022] POST /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451/action => generated 0 bytes in 85 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 24 08:53:08.739343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-5e0e7fce-9c6f-4f5d-90ee-5038cce2b764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:08.739951 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-5e0e7fce-9c6f-4f5d-90ee-5038cce2b764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:08.746325 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e0e7fce-9c6f-4f5d-90ee-5038cce2b764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:08.747020 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e0e7fce-9c6f-4f5d-90ee-5038cce2b764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:08.747390 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e0e7fce-9c6f-4f5d-90ee-5038cce2b764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:08.748085 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e0e7fce-9c6f-4f5d-90ee-5038cce2b764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:08.748565 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e0e7fce-9c6f-4f5d-90ee-5038cce2b764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:08.748988 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e0e7fce-9c6f-4f5d-90ee-5038cce2b764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:09.054485 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-5e0e7fce-9c6f-4f5d-90ee-5038cce2b764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:09.063968 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-5e0e7fce-9c6f-4f5d-90ee-5038cce2b764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1510 microversion: 2.1 time: 0.328416 Oct 24 08:53:09.065136 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 158/305] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:08 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1510 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:10.090906 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2e975985-dafd-40ea-95e3-882673355a97 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:10.091701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2e975985-dafd-40ea-95e3-882673355a97 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:10.098188 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2e975985-dafd-40ea-95e3-882673355a97 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:10.099040 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2e975985-dafd-40ea-95e3-882673355a97 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:10.099486 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2e975985-dafd-40ea-95e3-882673355a97 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:10.100234 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2e975985-dafd-40ea-95e3-882673355a97 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:10.100698 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2e975985-dafd-40ea-95e3-882673355a97 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:10.101128 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2e975985-dafd-40ea-95e3-882673355a97 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:10.360641 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2e975985-dafd-40ea-95e3-882673355a97 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:10.372278 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2e975985-dafd-40ea-95e3-882673355a97 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1510 microversion: 2.1 time: 0.286295 Oct 24 08:53:10.378052 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-58618962-5b00-4b07-a541-11fb0bbb896d req-6d281493-265f-4c73-a529-3083e9c1cfd1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "89a2295a-90e9-4fc1-be3b-a130f7e71528", "tag": "47ad5200-c751-4a26-a72e-3a49c982245c"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:53:10.385560 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 148/306] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:10 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1510 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:10.389685 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-58618962-5b00-4b07-a541-11fb0bbb896d req-6d281493-265f-4c73-a529-3083e9c1cfd1 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:10.390181 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-58618962-5b00-4b07-a541-11fb0bbb896d req-6d281493-265f-4c73-a529-3083e9c1cfd1 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:10.390609 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-58618962-5b00-4b07-a541-11fb0bbb896d req-6d281493-265f-4c73-a529-3083e9c1cfd1 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:10.404000 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-58618962-5b00-4b07-a541-11fb0bbb896d req-6d281493-265f-4c73-a529-3083e9c1cfd1 service nova] Creating event network-changed:47ad5200-c751-4a26-a72e-3a49c982245c for instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 on hv-cinder-maste Oct 24 08:53:10.409078 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-58618962-5b00-4b07-a541-11fb0bbb896d req-6d281493-265f-4c73-a529-3083e9c1cfd1 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034684 Oct 24 08:53:10.409683 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 159/307] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:53:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:53:11.397481 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-871d5908-8f33-445b-9492-3223f5763dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:11.398292 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-871d5908-8f33-445b-9492-3223f5763dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:11.405462 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-871d5908-8f33-445b-9492-3223f5763dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:11.405964 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-871d5908-8f33-445b-9492-3223f5763dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:11.406400 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-871d5908-8f33-445b-9492-3223f5763dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:11.407328 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-871d5908-8f33-445b-9492-3223f5763dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:11.407825 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-871d5908-8f33-445b-9492-3223f5763dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:11.408308 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-871d5908-8f33-445b-9492-3223f5763dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:11.552864 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-871d5908-8f33-445b-9492-3223f5763dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:11.561178 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-871d5908-8f33-445b-9492-3223f5763dc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1510 microversion: 2.1 time: 0.167611 Oct 24 08:53:11.561748 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 149/308] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:11 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1510 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:12.586011 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f2af5447-8e1f-49a8-ae5b-4421be2e0cc5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:12.586860 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f2af5447-8e1f-49a8-ae5b-4421be2e0cc5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:12.594655 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f2af5447-8e1f-49a8-ae5b-4421be2e0cc5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:12.595248 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f2af5447-8e1f-49a8-ae5b-4421be2e0cc5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:12.595760 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f2af5447-8e1f-49a8-ae5b-4421be2e0cc5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:12.596690 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f2af5447-8e1f-49a8-ae5b-4421be2e0cc5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:12.597191 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f2af5447-8e1f-49a8-ae5b-4421be2e0cc5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:12.597693 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f2af5447-8e1f-49a8-ae5b-4421be2e0cc5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:12.765815 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f2af5447-8e1f-49a8-ae5b-4421be2e0cc5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:12.774343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f2af5447-8e1f-49a8-ae5b-4421be2e0cc5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1510 microversion: 2.1 time: 0.192916 Oct 24 08:53:12.774937 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 160/309] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:12 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1510 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:13.801908 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-1f884c5a-a543-4fbe-888e-317c1838b30d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:13.802538 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-1f884c5a-a543-4fbe-888e-317c1838b30d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:13.809291 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1f884c5a-a543-4fbe-888e-317c1838b30d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:13.809782 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1f884c5a-a543-4fbe-888e-317c1838b30d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:13.810220 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1f884c5a-a543-4fbe-888e-317c1838b30d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:13.810928 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1f884c5a-a543-4fbe-888e-317c1838b30d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:13.811381 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1f884c5a-a543-4fbe-888e-317c1838b30d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:13.811800 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1f884c5a-a543-4fbe-888e-317c1838b30d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:13.962343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-1f884c5a-a543-4fbe-888e-317c1838b30d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:13.971884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-1f884c5a-a543-4fbe-888e-317c1838b30d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1510 microversion: 2.1 time: 0.176473 Oct 24 08:53:13.972507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 150/310] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:13 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1510 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:14.995877 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-bc94c9c5-7b62-4afc-9e7d-07be8e71a421 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:14.996892 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-bc94c9c5-7b62-4afc-9e7d-07be8e71a421 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:15.003171 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bc94c9c5-7b62-4afc-9e7d-07be8e71a421 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:15.003824 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bc94c9c5-7b62-4afc-9e7d-07be8e71a421 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:15.004373 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bc94c9c5-7b62-4afc-9e7d-07be8e71a421 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:15.005171 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bc94c9c5-7b62-4afc-9e7d-07be8e71a421 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:15.005688 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bc94c9c5-7b62-4afc-9e7d-07be8e71a421 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:15.006324 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bc94c9c5-7b62-4afc-9e7d-07be8e71a421 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:15.177562 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-bc94c9c5-7b62-4afc-9e7d-07be8e71a421 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:15.189601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-bc94c9c5-7b62-4afc-9e7d-07be8e71a421 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1516 microversion: 2.1 time: 0.197665 Oct 24 08:53:15.205145 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 161/311] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:14 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1516 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:22.942598 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-08156827-2f91-479f-99aa-a524dbff7f95 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Action: 'create', calling method: .version_select at 0x7fdfedf42f70>, body: {"keypair": {"name": "tempest-keypair-1226907291", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIcs8iCvtbuUTDltzJNqCrM37sK/GnGNPCfYK4L+DFf1Q9sdUKrlIYcYRM2vl1eJqQte3a0Sh4BmxJ4umau0CrfZO6VMmz3aWdWaX+Ln/xuwpS4wtoyGVOtO4ndmV4LsMg=="}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:53:22.948678 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=44 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:53:22.960828 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-08156827-2f91-479f-99aa-a524dbff7f95 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Getting quotas for project fd4b77e78f9d4286aa92f6f90ff8b797. Resources: {'key_pairs'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:53:22.966500 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-08156827-2f91-479f-99aa-a524dbff7f95 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Getting quotas for user 7127613bde784330bec364f02180291e and project fd4b77e78f9d4286aa92f6f90ff8b797. Resources: {'key_pairs'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:53:22.991299 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-08156827-2f91-479f-99aa-a524dbff7f95 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.052545 Oct 24 08:53:22.991987 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 151/312] 10.222.0.39 () {60 vars in 1159 bytes} [Mon Oct 24 08:53:22 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:53:23.609996 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1455483803", "imageRef": "bba331ba-faf5-4a56-81d9-6e84e9609665", "flavorRef": "452"}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:53:23.619239 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=46 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:53:23.667359 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.network.neutron [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] validate_networks() for None {{(pid=130395) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 08:53:24.000572 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Flavor limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 08:53:24.000988 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Image limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 08:53:24.001458 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Flavor pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 08:53:24.001860 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Image pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 08:53:24.002264 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 08:53:24.011136 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:24.012181 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:24.012618 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:24.013338 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:24.013787 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:24.014202 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:24.031702 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Getting quotas for project 047bb88ce8ab4fc9b4ce90aa89b3e7c8. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:53:24.036742 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Getting quotas for user 21e48a878e704c8f8625c1cd00474ec3 and project 047bb88ce8ab4fc9b4ce90aa89b3e7c8. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:53:24.050829 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Going to run 1 instances... {{(pid=130395) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 08:53:24.053201 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bba331ba-faf5-4a56-81d9-6e84e9609665',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130395) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 08:53:24.104697 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e805e88c-e5a8-4668-988b-a57074096020 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.499343 Oct 24 08:53:24.106002 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 162/313] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 08:53:23 2022] POST /compute/v2.1/servers => generated 380 bytes in 501 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 08:53:24.117629 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-438d3f9b-1770-4ad8-a127-3268b012f51f tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:24.118243 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-438d3f9b-1770-4ad8-a127-3268b012f51f tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:24.128347 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=90,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:53:24.188586 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-438d3f9b-1770-4ad8-a127-3268b012f51f tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21e48a878e704c8f8625c1cd00474ec3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '047bb88ce8ab4fc9b4ce90aa89b3e7c8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:24.198132 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-438d3f9b-1770-4ad8-a127-3268b012f51f tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 200 len: 1215 microversion: 2.1 time: 0.084426 Oct 24 08:53:24.199174 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 152/314] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:24 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 1215 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:24.457268 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1087925820", "imageRef": "bba331ba-faf5-4a56-81d9-6e84e9609665", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "51d12bec-f9d9-464c-a576-7d16d990480f"}], "security_groups": [{"name": "tempest-securitygroup--344959574"}], "key_name": "tempest-keypair-1226907291", "user_data": "}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:53:24.565533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.network.neutron [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] validate_networks() for [('51d12bec-f9d9-464c-a576-7d16d990480f', None, None, None, None, None)] {{(pid=130395) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 08:53:24.981655 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Flavor limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 08:53:24.982278 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Image limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 08:53:24.982528 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Flavor pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 08:53:24.982934 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Image pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 08:53:24.983344 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 08:53:24.997523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:24.998023 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:24.998468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:24.999207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:24.999666 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:25.000094 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:25.020099 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Getting quotas for project fd4b77e78f9d4286aa92f6f90ff8b797. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:53:25.024021 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Getting quotas for user 7127613bde784330bec364f02180291e and project fd4b77e78f9d4286aa92f6f90ff8b797. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:53:25.037778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Going to run 1 instances... {{(pid=130395) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 08:53:25.042012 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bba331ba-faf5-4a56-81d9-6e84e9609665',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130395) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 08:53:25.101095 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-6fe21297-dcdb-44f9-855c-5f4766569dcf tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 0.648315 Oct 24 08:53:25.101750 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 163/315] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 08:53:24 2022] POST /compute/v2.1/servers => generated 408 bytes in 650 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 08:53:25.115938 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-bb88202d-5d28-4efe-8a8e-a9fbb82125c1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:25.116924 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-bb88202d-5d28-4efe-8a8e-a9fbb82125c1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:25.203897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-bb88202d-5d28-4efe-8a8e-a9fbb82125c1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:25.212330 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-bb88202d-5d28-4efe-8a8e-a9fbb82125c1 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1246 microversion: 2.1 time: 0.100844 Oct 24 08:53:25.213061 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 153/316] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:25 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1246 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:25.228115 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-677a2667-978e-4594-aac1-a5b89987d42b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:25.228937 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-677a2667-978e-4594-aac1-a5b89987d42b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:25.236745 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-677a2667-978e-4594-aac1-a5b89987d42b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:25.237388 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-677a2667-978e-4594-aac1-a5b89987d42b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:25.238001 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-677a2667-978e-4594-aac1-a5b89987d42b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:25.238872 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-677a2667-978e-4594-aac1-a5b89987d42b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:25.239477 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-677a2667-978e-4594-aac1-a5b89987d42b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:25.240341 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-677a2667-978e-4594-aac1-a5b89987d42b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:25.312675 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-677a2667-978e-4594-aac1-a5b89987d42b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21e48a878e704c8f8625c1cd00474ec3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '047bb88ce8ab4fc9b4ce90aa89b3e7c8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:25.321130 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-677a2667-978e-4594-aac1-a5b89987d42b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 200 len: 1267 microversion: 2.1 time: 0.103269 Oct 24 08:53:25.321701 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 164/317] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:25 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 1267 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:26.251688 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9dd8bf89-0f83-43f1-8b1a-17f06d69af11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:26.253108 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9dd8bf89-0f83-43f1-8b1a-17f06d69af11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:26.260968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9dd8bf89-0f83-43f1-8b1a-17f06d69af11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:26.262086 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9dd8bf89-0f83-43f1-8b1a-17f06d69af11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:26.262680 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9dd8bf89-0f83-43f1-8b1a-17f06d69af11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:26.263385 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9dd8bf89-0f83-43f1-8b1a-17f06d69af11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:26.263846 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9dd8bf89-0f83-43f1-8b1a-17f06d69af11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:26.264311 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9dd8bf89-0f83-43f1-8b1a-17f06d69af11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:26.326917 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9dd8bf89-0f83-43f1-8b1a-17f06d69af11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:26.336291 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9dd8bf89-0f83-43f1-8b1a-17f06d69af11 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1298 microversion: 2.1 time: 0.088348 Oct 24 08:53:26.336858 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 154/318] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:26 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1298 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:26.346457 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-cce01c2c-ba1a-47e5-831b-c7514fd113f3 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:26.347208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-cce01c2c-ba1a-47e5-831b-c7514fd113f3 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:26.353697 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cce01c2c-ba1a-47e5-831b-c7514fd113f3 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:26.354490 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cce01c2c-ba1a-47e5-831b-c7514fd113f3 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:26.354873 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cce01c2c-ba1a-47e5-831b-c7514fd113f3 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:26.355747 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cce01c2c-ba1a-47e5-831b-c7514fd113f3 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:26.356238 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cce01c2c-ba1a-47e5-831b-c7514fd113f3 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:26.356671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cce01c2c-ba1a-47e5-831b-c7514fd113f3 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:26.425485 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cce01c2c-ba1a-47e5-831b-c7514fd113f3 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21e48a878e704c8f8625c1cd00474ec3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '047bb88ce8ab4fc9b4ce90aa89b3e7c8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:26.434869 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-cce01c2c-ba1a-47e5-831b-c7514fd113f3 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 200 len: 1273 microversion: 2.1 time: 0.092411 Oct 24 08:53:26.435456 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 165/319] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:26 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 1273 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:27.359916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6d004e40-1694-40c7-a2fd-5b1991b84fa4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:27.360521 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6d004e40-1694-40c7-a2fd-5b1991b84fa4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:27.367391 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d004e40-1694-40c7-a2fd-5b1991b84fa4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:27.368209 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d004e40-1694-40c7-a2fd-5b1991b84fa4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:27.368704 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d004e40-1694-40c7-a2fd-5b1991b84fa4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:27.369422 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d004e40-1694-40c7-a2fd-5b1991b84fa4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:27.369884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d004e40-1694-40c7-a2fd-5b1991b84fa4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:27.370312 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d004e40-1694-40c7-a2fd-5b1991b84fa4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:27.448059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6d004e40-1694-40c7-a2fd-5b1991b84fa4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:27.463372 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-acdd6eef-47d2-45d3-aa65-3234783ec8de tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:27.466011 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-acdd6eef-47d2-45d3-aa65-3234783ec8de tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:27.466236 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6d004e40-1694-40c7-a2fd-5b1991b84fa4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1304 microversion: 2.1 time: 0.110702 Oct 24 08:53:27.467165 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 155/320] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:27 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1304 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:27.485448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-acdd6eef-47d2-45d3-aa65-3234783ec8de tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:27.487046 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-acdd6eef-47d2-45d3-aa65-3234783ec8de tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:27.488082 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-acdd6eef-47d2-45d3-aa65-3234783ec8de tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:27.490532 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-acdd6eef-47d2-45d3-aa65-3234783ec8de tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:27.491406 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-acdd6eef-47d2-45d3-aa65-3234783ec8de tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:27.493073 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-acdd6eef-47d2-45d3-aa65-3234783ec8de tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:27.718461 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-acdd6eef-47d2-45d3-aa65-3234783ec8de tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21e48a878e704c8f8625c1cd00474ec3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '047bb88ce8ab4fc9b4ce90aa89b3e7c8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:27.724883 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-92001162-9d8e-4198-9fd6-b39824770051 req-cf0fe73c-3f02-4f50-b4c8-54509752d4e8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7b8871d0-526d-464c-b8a5-f168335a4185", "tag": "b7113ec6-d306-4869-9f8e-82ff435b1255"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:53:27.741224 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-92001162-9d8e-4198-9fd6-b39824770051 req-cf0fe73c-3f02-4f50-b4c8-54509752d4e8 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:27.742216 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-92001162-9d8e-4198-9fd6-b39824770051 req-cf0fe73c-3f02-4f50-b4c8-54509752d4e8 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:27.742903 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-92001162-9d8e-4198-9fd6-b39824770051 req-cf0fe73c-3f02-4f50-b4c8-54509752d4e8 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:27.746687 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-acdd6eef-47d2-45d3-aa65-3234783ec8de tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 200 len: 1315 microversion: 2.1 time: 0.284483 Oct 24 08:53:27.759903 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-92001162-9d8e-4198-9fd6-b39824770051 req-cf0fe73c-3f02-4f50-b4c8-54509752d4e8 service nova] Creating event network-changed:b7113ec6-d306-4869-9f8e-82ff435b1255 for instance 7b8871d0-526d-464c-b8a5-f168335a4185 on hv-cinder-maste Oct 24 08:53:27.763192 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 166/321] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:27 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 1315 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:27.766764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-92001162-9d8e-4198-9fd6-b39824770051 req-cf0fe73c-3f02-4f50-b4c8-54509752d4e8 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047188 Oct 24 08:53:27.767654 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 156/322] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:53:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:53:28.491037 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8ad1e7f0-9a09-4053-af41-e415767e568c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:28.491726 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8ad1e7f0-9a09-4053-af41-e415767e568c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:28.499816 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ad1e7f0-9a09-4053-af41-e415767e568c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:28.500596 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ad1e7f0-9a09-4053-af41-e415767e568c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:28.500979 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ad1e7f0-9a09-4053-af41-e415767e568c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:28.501735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ad1e7f0-9a09-4053-af41-e415767e568c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:28.502297 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ad1e7f0-9a09-4053-af41-e415767e568c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:28.502930 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ad1e7f0-9a09-4053-af41-e415767e568c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:28.713654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8ad1e7f0-9a09-4053-af41-e415767e568c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:28.721786 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8ad1e7f0-9a09-4053-af41-e415767e568c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1371 microversion: 2.1 time: 0.235443 Oct 24 08:53:28.722364 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 167/323] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:28 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1371 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:28.757376 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-34f5f3cd-5830-4cbf-bc71-01e2cfad9a50 req-1d9463d3-80b5-4a14-bf7c-72364956666f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf", "tag": "2a188a19-a722-46ea-96ea-89232fd07436"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:53:28.767326 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-34f5f3cd-5830-4cbf-bc71-01e2cfad9a50 req-1d9463d3-80b5-4a14-bf7c-72364956666f service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:28.768216 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-34f5f3cd-5830-4cbf-bc71-01e2cfad9a50 req-1d9463d3-80b5-4a14-bf7c-72364956666f service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:28.768902 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-34f5f3cd-5830-4cbf-bc71-01e2cfad9a50 req-1d9463d3-80b5-4a14-bf7c-72364956666f service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:28.773432 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-378787ef-c482-4267-9449-ab19991d866f tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:28.774059 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-378787ef-c482-4267-9449-ab19991d866f tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:28.780448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-378787ef-c482-4267-9449-ab19991d866f tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:28.780940 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-378787ef-c482-4267-9449-ab19991d866f tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:28.781379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-378787ef-c482-4267-9449-ab19991d866f tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:28.781695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-34f5f3cd-5830-4cbf-bc71-01e2cfad9a50 req-1d9463d3-80b5-4a14-bf7c-72364956666f service nova] Creating event network-changed:2a188a19-a722-46ea-96ea-89232fd07436 for instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf on hv-cinder-maste Oct 24 08:53:28.782095 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-378787ef-c482-4267-9449-ab19991d866f tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:28.782549 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-378787ef-c482-4267-9449-ab19991d866f tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:28.782975 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-378787ef-c482-4267-9449-ab19991d866f tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:28.787541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-34f5f3cd-5830-4cbf-bc71-01e2cfad9a50 req-1d9463d3-80b5-4a14-bf7c-72364956666f service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035056 Oct 24 08:53:28.788644 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 157/324] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:53:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:53:28.803626 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=100 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:53:28.924731 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-378787ef-c482-4267-9449-ab19991d866f tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21e48a878e704c8f8625c1cd00474ec3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '047bb88ce8ab4fc9b4ce90aa89b3e7c8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:28.932451 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-378787ef-c482-4267-9449-ab19991d866f tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 200 len: 1315 microversion: 2.1 time: 0.162970 Oct 24 08:53:28.933992 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 168/325] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:28 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 1315 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:29.746972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5096228e-9d73-41f5-a082-4bc02385c6bc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:29.747585 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-5096228e-9d73-41f5-a082-4bc02385c6bc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:29.755217 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5096228e-9d73-41f5-a082-4bc02385c6bc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:29.755704 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5096228e-9d73-41f5-a082-4bc02385c6bc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:29.756157 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5096228e-9d73-41f5-a082-4bc02385c6bc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:29.756851 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5096228e-9d73-41f5-a082-4bc02385c6bc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:29.757302 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5096228e-9d73-41f5-a082-4bc02385c6bc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:29.757721 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5096228e-9d73-41f5-a082-4bc02385c6bc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:29.958516 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d10c2806-0917-4c9e-a24c-76dd727fe192 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:29.959662 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d10c2806-0917-4c9e-a24c-76dd727fe192 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:29.967234 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d10c2806-0917-4c9e-a24c-76dd727fe192 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:29.967861 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d10c2806-0917-4c9e-a24c-76dd727fe192 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:29.968419 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d10c2806-0917-4c9e-a24c-76dd727fe192 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:29.970544 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d10c2806-0917-4c9e-a24c-76dd727fe192 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:29.972803 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d10c2806-0917-4c9e-a24c-76dd727fe192 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:29.973797 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d10c2806-0917-4c9e-a24c-76dd727fe192 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:30.216369 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5096228e-9d73-41f5-a082-4bc02385c6bc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:30.227166 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5096228e-9d73-41f5-a082-4bc02385c6bc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1371 microversion: 2.1 time: 0.484488 Oct 24 08:53:30.227927 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 158/326] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:29 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1371 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:30.234066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d10c2806-0917-4c9e-a24c-76dd727fe192 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21e48a878e704c8f8625c1cd00474ec3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '047bb88ce8ab4fc9b4ce90aa89b3e7c8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:30.243103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d10c2806-0917-4c9e-a24c-76dd727fe192 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 200 len: 1315 microversion: 2.1 time: 0.290744 Oct 24 08:53:30.243697 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 169/327] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:29 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 1315 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:31.252815 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b6b6b2a7-c6e6-42e2-8687-d46686b2519f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:31.254161 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b6b6b2a7-c6e6-42e2-8687-d46686b2519f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:31.263073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b6b6b2a7-c6e6-42e2-8687-d46686b2519f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:31.263893 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b6b6b2a7-c6e6-42e2-8687-d46686b2519f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:31.264587 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b6b6b2a7-c6e6-42e2-8687-d46686b2519f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:31.265514 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b6b6b2a7-c6e6-42e2-8687-d46686b2519f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:31.266598 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b6b6b2a7-c6e6-42e2-8687-d46686b2519f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:31.267854 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b6b6b2a7-c6e6-42e2-8687-d46686b2519f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:31.269374 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ae65b2e9-c148-49d1-baad-8ea4e89d7473 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:31.270183 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ae65b2e9-c148-49d1-baad-8ea4e89d7473 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:31.277865 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ae65b2e9-c148-49d1-baad-8ea4e89d7473 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:31.278886 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ae65b2e9-c148-49d1-baad-8ea4e89d7473 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:31.279503 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ae65b2e9-c148-49d1-baad-8ea4e89d7473 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:31.280778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ae65b2e9-c148-49d1-baad-8ea4e89d7473 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:31.281397 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ae65b2e9-c148-49d1-baad-8ea4e89d7473 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:31.281983 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ae65b2e9-c148-49d1-baad-8ea4e89d7473 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:31.569525 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b6b6b2a7-c6e6-42e2-8687-d46686b2519f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:31.578836 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b6b6b2a7-c6e6-42e2-8687-d46686b2519f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1371 microversion: 2.1 time: 0.330589 Oct 24 08:53:31.593002 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 159/328] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:31 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1371 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:31.613109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ae65b2e9-c148-49d1-baad-8ea4e89d7473 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21e48a878e704c8f8625c1cd00474ec3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '047bb88ce8ab4fc9b4ce90aa89b3e7c8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:31.624462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ae65b2e9-c148-49d1-baad-8ea4e89d7473 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 200 len: 1315 microversion: 2.1 time: 0.360548 Oct 24 08:53:31.625496 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 170/329] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:31 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 1315 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:32.616237 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a62595e3-619a-4dd5-90e3-d7c7bd416b67 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:32.619172 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-a62595e3-619a-4dd5-90e3-d7c7bd416b67 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:32.656428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a62595e3-619a-4dd5-90e3-d7c7bd416b67 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:32.657232 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a62595e3-619a-4dd5-90e3-d7c7bd416b67 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:32.658389 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a62595e3-619a-4dd5-90e3-d7c7bd416b67 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:32.660655 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a62595e3-619a-4dd5-90e3-d7c7bd416b67 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:32.661363 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a62595e3-619a-4dd5-90e3-d7c7bd416b67 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:32.663350 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a62595e3-619a-4dd5-90e3-d7c7bd416b67 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:32.758917 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-42ca4510-521c-4141-964b-2a87a79584d7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:32.761139 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-42ca4510-521c-4141-964b-2a87a79584d7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:32.770513 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42ca4510-521c-4141-964b-2a87a79584d7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:32.772226 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42ca4510-521c-4141-964b-2a87a79584d7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:32.772908 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42ca4510-521c-4141-964b-2a87a79584d7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:32.773782 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42ca4510-521c-4141-964b-2a87a79584d7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:32.774528 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42ca4510-521c-4141-964b-2a87a79584d7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:32.775030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-42ca4510-521c-4141-964b-2a87a79584d7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:32.946223 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-a62595e3-619a-4dd5-90e3-d7c7bd416b67 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:32.966947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-a62595e3-619a-4dd5-90e3-d7c7bd416b67 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1371 microversion: 2.1 time: 0.358098 Oct 24 08:53:32.968646 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 160/330] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:32 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1371 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:33.070417 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-42ca4510-521c-4141-964b-2a87a79584d7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21e48a878e704c8f8625c1cd00474ec3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '047bb88ce8ab4fc9b4ce90aa89b3e7c8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:33.080162 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-42ca4510-521c-4141-964b-2a87a79584d7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 200 len: 1315 microversion: 2.1 time: 0.325088 Oct 24 08:53:33.093442 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 171/331] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:32 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 1315 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:33.998168 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c36baab1-1aa8-477f-96ea-8193a8c26045 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:33.999505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c36baab1-1aa8-477f-96ea-8193a8c26045 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:34.007612 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c36baab1-1aa8-477f-96ea-8193a8c26045 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:34.010466 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c36baab1-1aa8-477f-96ea-8193a8c26045 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:34.012922 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c36baab1-1aa8-477f-96ea-8193a8c26045 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:34.014029 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c36baab1-1aa8-477f-96ea-8193a8c26045 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:34.015034 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c36baab1-1aa8-477f-96ea-8193a8c26045 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:34.015709 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c36baab1-1aa8-477f-96ea-8193a8c26045 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:34.107342 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-564a4977-ec88-4d61-8384-56d92a91ed6c tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:34.108828 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-564a4977-ec88-4d61-8384-56d92a91ed6c tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:34.118064 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-564a4977-ec88-4d61-8384-56d92a91ed6c tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:34.126055 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-564a4977-ec88-4d61-8384-56d92a91ed6c tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:34.126757 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-564a4977-ec88-4d61-8384-56d92a91ed6c tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:34.127797 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-564a4977-ec88-4d61-8384-56d92a91ed6c tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:34.128488 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-564a4977-ec88-4d61-8384-56d92a91ed6c tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:34.129340 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-564a4977-ec88-4d61-8384-56d92a91ed6c tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:34.340226 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c36baab1-1aa8-477f-96ea-8193a8c26045 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:34.360836 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c36baab1-1aa8-477f-96ea-8193a8c26045 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1371 microversion: 2.1 time: 0.362751 Oct 24 08:53:34.364727 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 161/332] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:33 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1371 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:34.471635 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-564a4977-ec88-4d61-8384-56d92a91ed6c tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21e48a878e704c8f8625c1cd00474ec3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '047bb88ce8ab4fc9b4ce90aa89b3e7c8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:34.487873 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-564a4977-ec88-4d61-8384-56d92a91ed6c tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 200 len: 1315 microversion: 2.1 time: 0.384689 Oct 24 08:53:34.490458 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 172/333] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:34 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 1315 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:35.391077 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-48e22085-bd97-41ad-b180-d5a4436b6b68 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:35.392286 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-48e22085-bd97-41ad-b180-d5a4436b6b68 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:35.400744 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=8 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:53:35.412669 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-48e22085-bd97-41ad-b180-d5a4436b6b68 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:35.413277 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-48e22085-bd97-41ad-b180-d5a4436b6b68 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:35.414028 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-48e22085-bd97-41ad-b180-d5a4436b6b68 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:35.415313 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-48e22085-bd97-41ad-b180-d5a4436b6b68 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:35.418088 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-48e22085-bd97-41ad-b180-d5a4436b6b68 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:35.418088 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-48e22085-bd97-41ad-b180-d5a4436b6b68 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:35.514322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8dadb3b2-bf59-438f-a7ce-6eb7f920f3b8 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:35.515227 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8dadb3b2-bf59-438f-a7ce-6eb7f920f3b8 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:35.523214 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=4 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:53:35.524423 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=12 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:53:35.527215 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8dadb3b2-bf59-438f-a7ce-6eb7f920f3b8 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:35.529335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8dadb3b2-bf59-438f-a7ce-6eb7f920f3b8 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:35.530121 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8dadb3b2-bf59-438f-a7ce-6eb7f920f3b8 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:35.531144 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8dadb3b2-bf59-438f-a7ce-6eb7f920f3b8 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:35.531614 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8dadb3b2-bf59-438f-a7ce-6eb7f920f3b8 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:35.532475 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8dadb3b2-bf59-438f-a7ce-6eb7f920f3b8 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:35.734971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-48e22085-bd97-41ad-b180-d5a4436b6b68 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:35.744796 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-48e22085-bd97-41ad-b180-d5a4436b6b68 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1371 microversion: 2.1 time: 0.362714 Oct 24 08:53:35.745905 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 162/334] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:35 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1371 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:35.799402 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8dadb3b2-bf59-438f-a7ce-6eb7f920f3b8 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21e48a878e704c8f8625c1cd00474ec3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '047bb88ce8ab4fc9b4ce90aa89b3e7c8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:35.827504 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8dadb3b2-bf59-438f-a7ce-6eb7f920f3b8 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 200 len: 1315 microversion: 2.1 time: 0.318182 Oct 24 08:53:35.842852 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 173/335] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:35 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 1315 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:36.268946 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-96bd58f0-ece6-4edb-9d47-e33115291ddb req-9d90bfe1-d10a-49c1-947a-496bb229f75b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7b8871d0-526d-464c-b8a5-f168335a4185", "name": "network-vif-plugged", "status": "completed", "tag": "b7113ec6-d306-4869-9f8e-82ff435b1255"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:53:36.279786 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-96bd58f0-ece6-4edb-9d47-e33115291ddb req-9d90bfe1-d10a-49c1-947a-496bb229f75b service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:36.281059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-96bd58f0-ece6-4edb-9d47-e33115291ddb req-9d90bfe1-d10a-49c1-947a-496bb229f75b service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:36.282829 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-96bd58f0-ece6-4edb-9d47-e33115291ddb req-9d90bfe1-d10a-49c1-947a-496bb229f75b service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:36.302943 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-96bd58f0-ece6-4edb-9d47-e33115291ddb req-9d90bfe1-d10a-49c1-947a-496bb229f75b service nova] Creating event network-vif-plugged:b7113ec6-d306-4869-9f8e-82ff435b1255 for instance 7b8871d0-526d-464c-b8a5-f168335a4185 on hv-cinder-maste Oct 24 08:53:36.310380 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-96bd58f0-ece6-4edb-9d47-e33115291ddb req-9d90bfe1-d10a-49c1-947a-496bb229f75b service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044654 Oct 24 08:53:36.311660 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 163/336] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:53:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:53:36.772275 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d83a8042-9b41-4e11-89a8-9813fbe26800 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:36.773724 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d83a8042-9b41-4e11-89a8-9813fbe26800 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:36.781415 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d83a8042-9b41-4e11-89a8-9813fbe26800 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:36.782160 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d83a8042-9b41-4e11-89a8-9813fbe26800 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:36.782862 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d83a8042-9b41-4e11-89a8-9813fbe26800 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:36.783800 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d83a8042-9b41-4e11-89a8-9813fbe26800 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:36.784447 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d83a8042-9b41-4e11-89a8-9813fbe26800 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:36.785036 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d83a8042-9b41-4e11-89a8-9813fbe26800 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:36.854667 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c17e5ba5-7b41-483c-8f1c-02155e7c7519 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:36.855800 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c17e5ba5-7b41-483c-8f1c-02155e7c7519 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:36.862995 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c17e5ba5-7b41-483c-8f1c-02155e7c7519 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:36.863817 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c17e5ba5-7b41-483c-8f1c-02155e7c7519 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:36.865381 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c17e5ba5-7b41-483c-8f1c-02155e7c7519 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:36.866331 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c17e5ba5-7b41-483c-8f1c-02155e7c7519 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:36.866962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c17e5ba5-7b41-483c-8f1c-02155e7c7519 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:36.867868 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c17e5ba5-7b41-483c-8f1c-02155e7c7519 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:37.117565 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d83a8042-9b41-4e11-89a8-9813fbe26800 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:37.132564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d83a8042-9b41-4e11-89a8-9813fbe26800 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1371 microversion: 2.1 time: 0.364286 Oct 24 08:53:37.137488 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 174/337] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:36 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1371 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:37.155056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c17e5ba5-7b41-483c-8f1c-02155e7c7519 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21e48a878e704c8f8625c1cd00474ec3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '047bb88ce8ab4fc9b4ce90aa89b3e7c8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:37.170096 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c17e5ba5-7b41-483c-8f1c-02155e7c7519 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 200 len: 1315 microversion: 2.1 time: 0.320127 Oct 24 08:53:37.170919 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 164/338] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:36 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 1315 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:38.165949 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f89c3eb9-302e-40b5-a48e-5428d307b5a4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:38.167002 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f89c3eb9-302e-40b5-a48e-5428d307b5a4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:38.177699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f89c3eb9-302e-40b5-a48e-5428d307b5a4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:38.178143 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f89c3eb9-302e-40b5-a48e-5428d307b5a4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:38.178972 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f89c3eb9-302e-40b5-a48e-5428d307b5a4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:38.179838 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f89c3eb9-302e-40b5-a48e-5428d307b5a4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:38.180322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f89c3eb9-302e-40b5-a48e-5428d307b5a4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:38.181035 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f89c3eb9-302e-40b5-a48e-5428d307b5a4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:38.200053 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9f419271-fb26-483a-a266-f76fa4c87107 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:38.201409 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9f419271-fb26-483a-a266-f76fa4c87107 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:38.208656 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9f419271-fb26-483a-a266-f76fa4c87107 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:38.209367 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9f419271-fb26-483a-a266-f76fa4c87107 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:38.210018 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9f419271-fb26-483a-a266-f76fa4c87107 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:38.210965 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9f419271-fb26-483a-a266-f76fa4c87107 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:38.211648 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9f419271-fb26-483a-a266-f76fa4c87107 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:38.212348 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9f419271-fb26-483a-a266-f76fa4c87107 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:38.502844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f89c3eb9-302e-40b5-a48e-5428d307b5a4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:38.509621 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9f419271-fb26-483a-a266-f76fa4c87107 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21e48a878e704c8f8625c1cd00474ec3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '047bb88ce8ab4fc9b4ce90aa89b3e7c8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:38.518659 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9f419271-fb26-483a-a266-f76fa4c87107 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 200 len: 1315 microversion: 2.1 time: 0.323438 Oct 24 08:53:38.519859 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 165/339] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:38 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 1315 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:38.523758 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f89c3eb9-302e-40b5-a48e-5428d307b5a4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1371 microversion: 2.1 time: 0.359937 Oct 24 08:53:38.536174 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 175/340] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:38 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1371 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:39.546750 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-21bc73a2-faed-455e-9423-2e34c75096ca tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:39.547303 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ea672f2e-6919-438d-8ea9-64a43df9ef69 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:39.547646 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-21bc73a2-faed-455e-9423-2e34c75096ca tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:39.547907 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ea672f2e-6919-438d-8ea9-64a43df9ef69 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:39.554230 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-21bc73a2-faed-455e-9423-2e34c75096ca tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:39.555092 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-21bc73a2-faed-455e-9423-2e34c75096ca tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:39.555796 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-21bc73a2-faed-455e-9423-2e34c75096ca tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:39.557405 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ea672f2e-6919-438d-8ea9-64a43df9ef69 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:39.558196 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ea672f2e-6919-438d-8ea9-64a43df9ef69 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:39.558383 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-21bc73a2-faed-455e-9423-2e34c75096ca tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:39.558783 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-21bc73a2-faed-455e-9423-2e34c75096ca tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:39.559480 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-21bc73a2-faed-455e-9423-2e34c75096ca tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:39.561029 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ea672f2e-6919-438d-8ea9-64a43df9ef69 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:39.561782 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ea672f2e-6919-438d-8ea9-64a43df9ef69 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:39.562868 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ea672f2e-6919-438d-8ea9-64a43df9ef69 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:39.563485 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ea672f2e-6919-438d-8ea9-64a43df9ef69 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:39.858333 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ea672f2e-6919-438d-8ea9-64a43df9ef69 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:39.867291 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-21bc73a2-faed-455e-9423-2e34c75096ca tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21e48a878e704c8f8625c1cd00474ec3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '047bb88ce8ab4fc9b4ce90aa89b3e7c8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:39.878657 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-21bc73a2-faed-455e-9423-2e34c75096ca tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 200 len: 1465 microversion: 2.1 time: 0.339858 Oct 24 08:53:39.880622 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 166/341] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:39 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 1465 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:39.889162 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ea672f2e-6919-438d-8ea9-64a43df9ef69 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1371 microversion: 2.1 time: 0.336015 Oct 24 08:53:39.889762 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 176/342] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:39 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1371 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:39.937819 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-37d312cc-53d0-4d22-9ee7-30c226644eef tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:39.938404 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-37d312cc-53d0-4d22-9ee7-30c226644eef tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:39.945292 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-37d312cc-53d0-4d22-9ee7-30c226644eef tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:39.946087 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-37d312cc-53d0-4d22-9ee7-30c226644eef tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:39.946773 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-37d312cc-53d0-4d22-9ee7-30c226644eef tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:39.947890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-37d312cc-53d0-4d22-9ee7-30c226644eef tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:39.948826 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-37d312cc-53d0-4d22-9ee7-30c226644eef tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:39.949512 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-37d312cc-53d0-4d22-9ee7-30c226644eef tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:40.003116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-37d312cc-53d0-4d22-9ee7-30c226644eef tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Going to try to terminate instance {{(pid=130396) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 08:53:40.006626 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-703f87cb-768b-4b57-8186-2bb20c677ee3 req-f2587adc-12ea-4b3c-af7a-688c0f616769 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf", "name": "network-vif-plugged", "status": "completed", "tag": "2a188a19-a722-46ea-96ea-89232fd07436"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:53:40.022853 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-703f87cb-768b-4b57-8186-2bb20c677ee3 req-f2587adc-12ea-4b3c-af7a-688c0f616769 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:40.023412 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-703f87cb-768b-4b57-8186-2bb20c677ee3 req-f2587adc-12ea-4b3c-af7a-688c0f616769 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:40.023901 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-703f87cb-768b-4b57-8186-2bb20c677ee3 req-f2587adc-12ea-4b3c-af7a-688c0f616769 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:40.073071 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-703f87cb-768b-4b57-8186-2bb20c677ee3 req-f2587adc-12ea-4b3c-af7a-688c0f616769 service nova] Creating event network-vif-plugged:2a188a19-a722-46ea-96ea-89232fd07436 for instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf on hv-cinder-maste Oct 24 08:53:40.093449 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-703f87cb-768b-4b57-8186-2bb20c677ee3 req-f2587adc-12ea-4b3c-af7a-688c0f616769 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.090536 Oct 24 08:53:40.096291 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 177/343] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:53:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:53:40.196293 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-37d312cc-53d0-4d22-9ee7-30c226644eef tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "DELETE /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 204 len: 0 microversion: 2.1 time: 0.261923 Oct 24 08:53:40.197589 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 167/344] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 08:53:39 2022] DELETE /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 08:53:40.207052 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8aee4bf0-ffdf-4530-998d-3bb943ee0ad7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:40.208000 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8aee4bf0-ffdf-4530-998d-3bb943ee0ad7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:40.219506 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8aee4bf0-ffdf-4530-998d-3bb943ee0ad7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:40.221566 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8aee4bf0-ffdf-4530-998d-3bb943ee0ad7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:40.222194 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8aee4bf0-ffdf-4530-998d-3bb943ee0ad7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:40.223097 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8aee4bf0-ffdf-4530-998d-3bb943ee0ad7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:40.223555 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8aee4bf0-ffdf-4530-998d-3bb943ee0ad7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:40.223979 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8aee4bf0-ffdf-4530-998d-3bb943ee0ad7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:40.384045 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8aee4bf0-ffdf-4530-998d-3bb943ee0ad7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21e48a878e704c8f8625c1cd00474ec3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '047bb88ce8ab4fc9b4ce90aa89b3e7c8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:40.393249 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8aee4bf0-ffdf-4530-998d-3bb943ee0ad7 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 200 len: 1471 microversion: 2.1 time: 0.190579 Oct 24 08:53:40.394548 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 178/345] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:40 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 1471 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:40.924434 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6a42ae95-23f8-4acb-b040-41a4d21b3546 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:40.925503 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6a42ae95-23f8-4acb-b040-41a4d21b3546 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:40.937981 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a42ae95-23f8-4acb-b040-41a4d21b3546 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:40.938488 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a42ae95-23f8-4acb-b040-41a4d21b3546 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:40.942470 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a42ae95-23f8-4acb-b040-41a4d21b3546 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:40.943426 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a42ae95-23f8-4acb-b040-41a4d21b3546 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:40.943918 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a42ae95-23f8-4acb-b040-41a4d21b3546 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:40.944398 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a42ae95-23f8-4acb-b040-41a4d21b3546 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:41.419182 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-2a1e5371-f993-4b15-a3fa-a71d2b08c90a tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:41.419807 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-2a1e5371-f993-4b15-a3fa-a71d2b08c90a tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:41.430932 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2a1e5371-f993-4b15-a3fa-a71d2b08c90a tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:41.431379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2a1e5371-f993-4b15-a3fa-a71d2b08c90a tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:41.432024 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2a1e5371-f993-4b15-a3fa-a71d2b08c90a tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:41.439021 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2a1e5371-f993-4b15-a3fa-a71d2b08c90a tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:41.439530 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2a1e5371-f993-4b15-a3fa-a71d2b08c90a tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:41.440011 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2a1e5371-f993-4b15-a3fa-a71d2b08c90a tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:41.754674 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6a42ae95-23f8-4acb-b040-41a4d21b3546 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:41.766029 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6a42ae95-23f8-4acb-b040-41a4d21b3546 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1371 microversion: 2.1 time: 0.845856 Oct 24 08:53:41.767741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 168/346] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:40 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1371 bytes in 849 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:41.831355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-2a1e5371-f993-4b15-a3fa-a71d2b08c90a tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21e48a878e704c8f8625c1cd00474ec3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '047bb88ce8ab4fc9b4ce90aa89b3e7c8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:41.839883 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-2a1e5371-f993-4b15-a3fa-a71d2b08c90a tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 200 len: 1471 microversion: 2.1 time: 0.425596 Oct 24 08:53:41.845826 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 179/347] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:41 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 1471 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:42.795038 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-216c384d-7e61-481f-ab1b-3fe16d6d37d4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:42.795668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-216c384d-7e61-481f-ab1b-3fe16d6d37d4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:42.802301 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-216c384d-7e61-481f-ab1b-3fe16d6d37d4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:42.802788 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-216c384d-7e61-481f-ab1b-3fe16d6d37d4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:42.803224 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-216c384d-7e61-481f-ab1b-3fe16d6d37d4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:42.804125 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-216c384d-7e61-481f-ab1b-3fe16d6d37d4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:42.804596 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-216c384d-7e61-481f-ab1b-3fe16d6d37d4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:42.805026 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-216c384d-7e61-481f-ab1b-3fe16d6d37d4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:42.871860 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1dc95ee4-b8aa-40e4-9e78-064f3028c727 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:42.872500 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1dc95ee4-b8aa-40e4-9e78-064f3028c727 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:42.883915 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1dc95ee4-b8aa-40e4-9e78-064f3028c727 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:42.885614 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1dc95ee4-b8aa-40e4-9e78-064f3028c727 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:42.886271 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1dc95ee4-b8aa-40e4-9e78-064f3028c727 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:42.887194 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1dc95ee4-b8aa-40e4-9e78-064f3028c727 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:42.889213 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1dc95ee4-b8aa-40e4-9e78-064f3028c727 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:42.889836 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1dc95ee4-b8aa-40e4-9e78-064f3028c727 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:43.108185 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-216c384d-7e61-481f-ab1b-3fe16d6d37d4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:43.123653 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-216c384d-7e61-481f-ab1b-3fe16d6d37d4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1371 microversion: 2.1 time: 0.333585 Oct 24 08:53:43.125099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 169/348] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:42 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1371 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:53:43.271740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1dc95ee4-b8aa-40e4-9e78-064f3028c727 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21e48a878e704c8f8625c1cd00474ec3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '047bb88ce8ab4fc9b4ce90aa89b3e7c8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:43.302132 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1dc95ee4-b8aa-40e4-9e78-064f3028c727 tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 200 len: 1471 microversion: 2.1 time: 0.434599 Oct 24 08:53:43.308963 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 180/349] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:42 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 1471 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:43.528184 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-ba5b1c47-ee43-44de-bb2c-e7d6a4c62c6f req-06a7673e-5d2c-4f4c-b763-a63e7f2f07dd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7b8871d0-526d-464c-b8a5-f168335a4185", "name": "network-vif-deleted", "tag": "b7113ec6-d306-4869-9f8e-82ff435b1255"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:53:43.540954 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-ba5b1c47-ee43-44de-bb2c-e7d6a4c62c6f req-06a7673e-5d2c-4f4c-b763-a63e7f2f07dd service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:43.541394 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-ba5b1c47-ee43-44de-bb2c-e7d6a4c62c6f req-06a7673e-5d2c-4f4c-b763-a63e7f2f07dd service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:43.541925 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-ba5b1c47-ee43-44de-bb2c-e7d6a4c62c6f req-06a7673e-5d2c-4f4c-b763-a63e7f2f07dd service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:43.554037 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-ba5b1c47-ee43-44de-bb2c-e7d6a4c62c6f req-06a7673e-5d2c-4f4c-b763-a63e7f2f07dd service nova] Creating event network-vif-deleted:b7113ec6-d306-4869-9f8e-82ff435b1255 for instance 7b8871d0-526d-464c-b8a5-f168335a4185 on hv-cinder-maste Oct 24 08:53:43.578736 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-ba5b1c47-ee43-44de-bb2c-e7d6a4c62c6f req-06a7673e-5d2c-4f4c-b763-a63e7f2f07dd service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044961 Oct 24 08:53:43.579431 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 170/350] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:53:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:53:44.104156 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-6fcdcc2a-1cc4-43f3-a52f-2ed7188e5692 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Action: 'create', calling method: .version_select at 0x7fdfedf06b80>, body: {"keypair": {"name": "tempest-TestShelveInstance-1609252738", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM9qVXQRcQzTd5US910GlS3KZ8lyWiDeWS4+LYJ5GMOY8+tKhnXKi8i10BppbYLBEaErKLVRb80R/qwta0T17o8jKKfUAiaqjv3BnCHZJLJEn2mu3WCmJNf79CjEbjFiLA=="}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:53:44.121894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-6fcdcc2a-1cc4-43f3-a52f-2ed7188e5692 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Getting quotas for project 29e4576f2b744d40878d65470e24c24c. Resources: {'key_pairs'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:53:44.126272 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-6fcdcc2a-1cc4-43f3-a52f-2ed7188e5692 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Getting quotas for user 82055ff074d842e88e27a441c32e9d01 and project 29e4576f2b744d40878d65470e24c24c. Resources: {'key_pairs'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:53:44.147709 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6141911c-8f81-473d-a41e-864a5836cbe4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:44.148939 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6141911c-8f81-473d-a41e-864a5836cbe4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:44.155512 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-6fcdcc2a-1cc4-43f3-a52f-2ed7188e5692 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.309969 Oct 24 08:53:44.156149 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 181/351] 10.222.0.39 () {60 vars in 1159 bytes} [Mon Oct 24 08:53:43 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:53:44.157267 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6141911c-8f81-473d-a41e-864a5836cbe4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:44.157740 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6141911c-8f81-473d-a41e-864a5836cbe4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:44.158181 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6141911c-8f81-473d-a41e-864a5836cbe4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:44.158888 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6141911c-8f81-473d-a41e-864a5836cbe4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:44.159342 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6141911c-8f81-473d-a41e-864a5836cbe4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:44.159771 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6141911c-8f81-473d-a41e-864a5836cbe4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:44.178795 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:53:44.329411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8f91124d-2170-4c5b-9af6-9fd28db0d95b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:44.330303 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8f91124d-2170-4c5b-9af6-9fd28db0d95b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:44.336478 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8f91124d-2170-4c5b-9af6-9fd28db0d95b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:44.336962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8f91124d-2170-4c5b-9af6-9fd28db0d95b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:44.337584 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8f91124d-2170-4c5b-9af6-9fd28db0d95b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:44.338286 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8f91124d-2170-4c5b-9af6-9fd28db0d95b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:44.339299 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8f91124d-2170-4c5b-9af6-9fd28db0d95b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:44.340031 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8f91124d-2170-4c5b-9af6-9fd28db0d95b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:44.350731 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6141911c-8f81-473d-a41e-864a5836cbe4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:44.360122 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6141911c-8f81-473d-a41e-864a5836cbe4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1565 microversion: 2.1 time: 0.216960 Oct 24 08:53:44.374007 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8f91124d-2170-4c5b-9af6-9fd28db0d95b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:44.374261 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 171/352] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:44 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1565 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:44.375016 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8f91124d-2170-4c5b-9af6-9fd28db0d95b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:44.375677 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8f91124d-2170-4c5b-9af6-9fd28db0d95b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:44.376313 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-8f91124d-2170-4c5b-9af6-9fd28db0d95b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lazy-loading 'fault' on Instance uuid 7b8871d0-526d-464c-b8a5-f168335a4185 {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:53:44.385137 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-47723c29-2c09-4931-8d8b-2af2a276358a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:44.386665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-47723c29-2c09-4931-8d8b-2af2a276358a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:44.393264 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47723c29-2c09-4931-8d8b-2af2a276358a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:44.393750 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47723c29-2c09-4931-8d8b-2af2a276358a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:44.394185 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47723c29-2c09-4931-8d8b-2af2a276358a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:44.394880 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47723c29-2c09-4931-8d8b-2af2a276358a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:44.395329 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47723c29-2c09-4931-8d8b-2af2a276358a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:44.395747 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47723c29-2c09-4931-8d8b-2af2a276358a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:44.446071 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8f91124d-2170-4c5b-9af6-9fd28db0d95b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21e48a878e704c8f8625c1cd00474ec3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '047bb88ce8ab4fc9b4ce90aa89b3e7c8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:53:44.457550 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8f91124d-2170-4c5b-9af6-9fd28db0d95b tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 200 len: 1305 microversion: 2.1 time: 0.132396 Oct 24 08:53:44.467316 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 182/353] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:44 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 1305 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:53:44.512948 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-47723c29-2c09-4931-8d8b-2af2a276358a tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.131745 Oct 24 08:53:44.513556 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 172/354] 10.222.0.39 () {58 vars in 1275 bytes} [Mon Oct 24 08:53:44 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/os-interface => generated 284 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:53:45.483347 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a4e79861-8607-4bb1-b4e8-61605254f0dc tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:53:45.483967 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a4e79861-8607-4bb1-b4e8-61605254f0dc tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] [instance: 7b8871d0-526d-464c-b8a5-f168335a4185] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:53:45.493467 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4e79861-8607-4bb1-b4e8-61605254f0dc tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:45.499007 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4e79861-8607-4bb1-b4e8-61605254f0dc tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:45.499464 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4e79861-8607-4bb1-b4e8-61605254f0dc tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:45.500232 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4e79861-8607-4bb1-b4e8-61605254f0dc tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:45.500841 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4e79861-8607-4bb1-b4e8-61605254f0dc tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:45.501415 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4e79861-8607-4bb1-b4e8-61605254f0dc tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:45.521780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-a4e79861-8607-4bb1-b4e8-61605254f0dc tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] HTTP exception thrown: Instance 7b8871d0-526d-464c-b8a5-f168335a4185 could not be found. Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a4e79861-8607-4bb1-b4e8-61605254f0dc tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Request method failure captured: Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 HTTP/1.1 Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlJ9Nse77Unv_QTYVtI0s_pRQw-jcBIcO96nWU1JbPlcZ_cyB0mPyHX_I8O6VIM6-HMIzsLmA4SE_hmaIwyJFaeyZOCA8EdfeJ6AfYkcxsTvOoo1JL6UXfHeja2w8D-wgpE3lCXq2RZUk7veA95gL73ksCr1DOeSDrdyggaPBxW5il4 Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: 047bb88ce8ab4fc9b4ce90aa89b3e7c8 Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-VolumesNegativeTest-1600403964 Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 08:53:45.523780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 08:53:45.525319 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/047bb88ce8ab4fc9b4ce90aa89b3e7c8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_047bb88ce8ab4fc9b4ce90aa89b3e7c8", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/047bb88ce8ab4fc9b4ce90aa89b3e7c8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/047bb88ce8ab4fc9b4ce90aa89b3e7c8"}]}] Oct 24 08:53:45.525319 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-VolumesNegativeTest-1600403964 Oct 24 08:53:45.525319 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: 047bb88ce8ab4fc9b4ce90aa89b3e7c8 Oct 24 08:53:45.525319 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-VolumesNegativeTest-1600403964 Oct 24 08:53:45.525319 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-VolumesNegativeTest-1600403964-project Oct 24 08:53:45.525319 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 08:53:45.525319 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 08:53:45.525319 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: 21e48a878e704c8f8625c1cd00474ec3 Oct 24 08:53:45.525319 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-VolumesNegativeTest-1600403964-project Oct 24 08:53:45.525319 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 08:53:45.525319 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance 7b8871d0-526d-464c-b8a5-f168335a4185 could not be found. Oct 24 08:53:45.525319 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': '7b8871d0-526d-464c-b8a5-f168335a4185'} Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7b8871d0-526d-464c-b8a5-f168335a4185 could not be found. Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:53:45.527259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7b8871d0-526d-464c-b8a5-f168335a4185 could not be found. Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7b8871d0-526d-464c-b8a5-f168335a4185 could not be found. Oct 24 08:53:45.534093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:53:45.536629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a4e79861-8607-4bb1-b4e8-61605254f0dc tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] Returning 404 to user: Instance 7b8871d0-526d-464c-b8a5-f168335a4185 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:53:45.536629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a4e79861-8607-4bb1-b4e8-61605254f0dc tempest-VolumesNegativeTest-1600403964 tempest-VolumesNegativeTest-1600403964-project] 10.222.0.39 "GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185" status: 404 len: 111 microversion: 2.1 time: 0.046783 Oct 24 08:53:45.536629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 183/355] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:53:45 2022] GET /compute/v2.1/servers/7b8871d0-526d-464c-b8a5-f168335a4185 => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 08:53:45.890851 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-41a92b52-30a1-4d24-9569-27a8ea987e9c req-7d15d5dc-33d5-4b06-8147-30d2b7204d05 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf", "tag": "2a188a19-a722-46ea-96ea-89232fd07436"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:53:45.908650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-41a92b52-30a1-4d24-9569-27a8ea987e9c req-7d15d5dc-33d5-4b06-8147-30d2b7204d05 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:53:45.909319 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-41a92b52-30a1-4d24-9569-27a8ea987e9c req-7d15d5dc-33d5-4b06-8147-30d2b7204d05 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:53:45.909898 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-41a92b52-30a1-4d24-9569-27a8ea987e9c req-7d15d5dc-33d5-4b06-8147-30d2b7204d05 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:53:45.922638 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-41a92b52-30a1-4d24-9569-27a8ea987e9c req-7d15d5dc-33d5-4b06-8147-30d2b7204d05 service nova] Creating event network-changed:2a188a19-a722-46ea-96ea-89232fd07436 for instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf on hv-cinder-maste Oct 24 08:53:45.927939 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-41a92b52-30a1-4d24-9569-27a8ea987e9c req-7d15d5dc-33d5-4b06-8147-30d2b7204d05 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045914 Oct 24 08:53:45.931870 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 173/356] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:53:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:54:09.708036 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-744028911", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1609252738", "security_groups": [{"name": "tempest-secgroup-smoke-1449348362"}], "networks": [{"uuid": "85c2e3a3-c6f8-4928-864d-45368094fcdf"}], "block_device_mapping_v2": [{"uuid": "955cf4ce-78a5-480e-b0cf-50cde6def5c7", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:54:09.719272 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=4 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:09.719991 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=78,nova_api:INSERT=7 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:09.720601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=81 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:09.825257 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.network.neutron [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] validate_networks() for [('85c2e3a3-c6f8-4928-864d-45368094fcdf', None, None, None, None, None)] {{(pid=130395) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 08:54:10.211497 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Flavor limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 08:54:10.211922 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Image limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 08:54:10.212443 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Flavor pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 08:54:10.213620 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Image pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 08:54:10.214220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 08:54:10.224251 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:10.224735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:10.225174 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:10.225906 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:10.226357 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:10.226791 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:10.244438 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Getting quotas for project 29e4576f2b744d40878d65470e24c24c. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:54:10.248690 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Getting quotas for user 82055ff074d842e88e27a441c32e9d01 and project 29e4576f2b744d40878d65470e24c24c. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:54:10.306864 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Going to run 1 instances... {{(pid=130395) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 08:54:10.309065 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='955cf4ce-78a5-480e-b0cf-50cde6def5c7',volume_size=None,volume_type=None)] {{(pid=130395) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 08:54:10.486150 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7e0a0266-a451-4fce-bfdb-bc4078e54728 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.781976 Oct 24 08:54:10.486702 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 184/357] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 08:54:09 2022] POST /compute/v2.1/servers => generated 406 bytes in 783 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 08:54:10.497686 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=63,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:10.498536 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=8 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:10.503708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8ad0aab1-83a3-42f4-b549-2acde0f17c52 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:10.504489 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8ad0aab1-83a3-42f4-b549-2acde0f17c52 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:10.565755 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8ad0aab1-83a3-42f4-b549-2acde0f17c52 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:10.573573 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8ad0aab1-83a3-42f4-b549-2acde0f17c52 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1088 microversion: 2.1 time: 0.078042 Oct 24 08:54:10.574161 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 174/358] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:10 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1088 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:11.657452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8361b8ab-3ea7-4561-bb1f-10d9f6c0184d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:11.658072 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8361b8ab-3ea7-4561-bb1f-10d9f6c0184d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:11.664764 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8361b8ab-3ea7-4561-bb1f-10d9f6c0184d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:11.665262 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8361b8ab-3ea7-4561-bb1f-10d9f6c0184d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:11.666195 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8361b8ab-3ea7-4561-bb1f-10d9f6c0184d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:11.666989 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8361b8ab-3ea7-4561-bb1f-10d9f6c0184d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:11.667450 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8361b8ab-3ea7-4561-bb1f-10d9f6c0184d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:11.667872 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8361b8ab-3ea7-4561-bb1f-10d9f6c0184d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:11.735590 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8361b8ab-3ea7-4561-bb1f-10d9f6c0184d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:11.743710 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8361b8ab-3ea7-4561-bb1f-10d9f6c0184d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1186 microversion: 2.1 time: 0.090022 Oct 24 08:54:11.744994 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 185/359] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:11 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1186 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:12.770928 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f8b3e0fd-33a8-450f-8ba9-cd085fd4cd88 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:12.771743 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f8b3e0fd-33a8-450f-8ba9-cd085fd4cd88 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:12.780079 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f8b3e0fd-33a8-450f-8ba9-cd085fd4cd88 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:12.780600 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f8b3e0fd-33a8-450f-8ba9-cd085fd4cd88 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:12.781125 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f8b3e0fd-33a8-450f-8ba9-cd085fd4cd88 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:12.781804 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f8b3e0fd-33a8-450f-8ba9-cd085fd4cd88 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:12.782266 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f8b3e0fd-33a8-450f-8ba9-cd085fd4cd88 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:12.782759 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f8b3e0fd-33a8-450f-8ba9-cd085fd4cd88 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:12.879627 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f8b3e0fd-33a8-450f-8ba9-cd085fd4cd88 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:12.888252 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f8b3e0fd-33a8-450f-8ba9-cd085fd4cd88 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1204 microversion: 2.1 time: 0.122447 Oct 24 08:54:12.888849 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 175/360] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:12 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1204 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:54:12.940697 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-79787a4a-2e1e-4be7-9272-cebfe1b3ba3a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:12.941368 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-79787a4a-2e1e-4be7-9272-cebfe1b3ba3a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:12.948341 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-79787a4a-2e1e-4be7-9272-cebfe1b3ba3a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:12.948896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-79787a4a-2e1e-4be7-9272-cebfe1b3ba3a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:12.949275 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-79787a4a-2e1e-4be7-9272-cebfe1b3ba3a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:12.950106 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-79787a4a-2e1e-4be7-9272-cebfe1b3ba3a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:12.950573 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-79787a4a-2e1e-4be7-9272-cebfe1b3ba3a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:12.950992 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-79787a4a-2e1e-4be7-9272-cebfe1b3ba3a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:12.981141 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-79787a4a-2e1e-4be7-9272-cebfe1b3ba3a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/ips" status: 200 len: 153 microversion: 2.20 time: 0.044499 Oct 24 08:54:12.982027 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 186/361] 10.222.0.39 () {60 vars in 1289 bytes} [Mon Oct 24 08:54:12 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/ips => generated 153 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:54:13.912496 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e5828c58-642a-4ee6-86c2-4539610ed094 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:13.913106 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e5828c58-642a-4ee6-86c2-4539610ed094 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:13.919902 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e5828c58-642a-4ee6-86c2-4539610ed094 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:13.920435 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e5828c58-642a-4ee6-86c2-4539610ed094 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:13.920870 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e5828c58-642a-4ee6-86c2-4539610ed094 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:13.921575 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e5828c58-642a-4ee6-86c2-4539610ed094 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:13.922025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e5828c58-642a-4ee6-86c2-4539610ed094 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:13.922447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e5828c58-642a-4ee6-86c2-4539610ed094 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:14.037906 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-3264d251-b9d3-4e1f-bd32-416c53fdff56 req-249dadd1-a4f1-4efe-a07f-fc2571c59280 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "354e90c0-1ef5-43e0-8983-2067982a3f76", "tag": "4a9dc16c-b094-43ae-a495-1098e5c9f054"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:54:14.050565 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-3264d251-b9d3-4e1f-bd32-416c53fdff56 req-249dadd1-a4f1-4efe-a07f-fc2571c59280 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:14.051048 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-3264d251-b9d3-4e1f-bd32-416c53fdff56 req-249dadd1-a4f1-4efe-a07f-fc2571c59280 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:14.051488 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-3264d251-b9d3-4e1f-bd32-416c53fdff56 req-249dadd1-a4f1-4efe-a07f-fc2571c59280 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:14.064393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e5828c58-642a-4ee6-86c2-4539610ed094 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:14.064706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-3264d251-b9d3-4e1f-bd32-416c53fdff56 req-249dadd1-a4f1-4efe-a07f-fc2571c59280 service nova] Creating event network-changed:4a9dc16c-b094-43ae-a495-1098e5c9f054 for instance 354e90c0-1ef5-43e0-8983-2067982a3f76 on hv-cinder-maste Oct 24 08:54:14.069872 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-3264d251-b9d3-4e1f-bd32-416c53fdff56 req-249dadd1-a4f1-4efe-a07f-fc2571c59280 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036916 Oct 24 08:54:14.070635 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 187/362] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:54:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:54:14.073571 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e5828c58-642a-4ee6-86c2-4539610ed094 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1260 microversion: 2.1 time: 0.164825 Oct 24 08:54:14.074236 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 176/363] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:13 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1260 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:15.034024 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1e8ac15a-a261-4802-be01-09058bb1d5ca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:54:15.034652 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1e8ac15a-a261-4802-be01-09058bb1d5ca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:15.041463 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1e8ac15a-a261-4802-be01-09058bb1d5ca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:15.042075 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1e8ac15a-a261-4802-be01-09058bb1d5ca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:15.042414 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1e8ac15a-a261-4802-be01-09058bb1d5ca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:15.043110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1e8ac15a-a261-4802-be01-09058bb1d5ca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:15.043563 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1e8ac15a-a261-4802-be01-09058bb1d5ca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:15.043991 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1e8ac15a-a261-4802-be01-09058bb1d5ca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:15.067423 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-1e8ac15a-a261-4802-be01-09058bb1d5ca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lazy-loading 'flavor' on Instance uuid 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:54:15.097338 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e898a1de-ff3f-4b8d-ab7e-b74a1e7f6384 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:15.098024 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e898a1de-ff3f-4b8d-ab7e-b74a1e7f6384 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:15.105154 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e898a1de-ff3f-4b8d-ab7e-b74a1e7f6384 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:15.105644 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e898a1de-ff3f-4b8d-ab7e-b74a1e7f6384 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:15.106079 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e898a1de-ff3f-4b8d-ab7e-b74a1e7f6384 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:15.107140 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e898a1de-ff3f-4b8d-ab7e-b74a1e7f6384 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:15.107337 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e898a1de-ff3f-4b8d-ab7e-b74a1e7f6384 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:15.107659 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e898a1de-ff3f-4b8d-ab7e-b74a1e7f6384 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:15.249691 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e898a1de-ff3f-4b8d-ab7e-b74a1e7f6384 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:15.257758 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e898a1de-ff3f-4b8d-ab7e-b74a1e7f6384 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1260 microversion: 2.1 time: 0.164289 Oct 24 08:54:15.258637 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 177/364] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:15 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1260 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:54:15.285429 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1e8ac15a-a261-4802-be01-09058bb1d5ca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "POST /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/action" status: 202 len: 0 microversion: 2.20 time: 0.255587 Oct 24 08:54:15.286021 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 188/365] 10.222.0.39 () {62 vars in 1319 bytes} [Mon Oct 24 08:54:15 2022] POST /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/action => generated 0 bytes in 257 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 24 08:54:15.294274 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4fc2112f-c711-4192-bee4-9e78f8c91a96 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:15.294918 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4fc2112f-c711-4192-bee4-9e78f8c91a96 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:15.301204 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4fc2112f-c711-4192-bee4-9e78f8c91a96 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:15.301694 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4fc2112f-c711-4192-bee4-9e78f8c91a96 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:15.302136 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4fc2112f-c711-4192-bee4-9e78f8c91a96 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:15.302836 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4fc2112f-c711-4192-bee4-9e78f8c91a96 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:15.303287 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4fc2112f-c711-4192-bee4-9e78f8c91a96 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:15.303720 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4fc2112f-c711-4192-bee4-9e78f8c91a96 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:15.437385 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4fc2112f-c711-4192-bee4-9e78f8c91a96 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:15.445810 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4fc2112f-c711-4192-bee4-9e78f8c91a96 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:15.448231 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4fc2112f-c711-4192-bee4-9e78f8c91a96 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:15.449019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4fc2112f-c711-4192-bee4-9e78f8c91a96 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1723 microversion: 2.20 time: 0.158633 Oct 24 08:54:15.449572 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 178/366] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:15 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1723 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:16.282694 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c7240901-4f6d-424f-acb0-038a3e238a4f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:16.283319 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c7240901-4f6d-424f-acb0-038a3e238a4f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:16.289664 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7240901-4f6d-424f-acb0-038a3e238a4f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:16.290178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7240901-4f6d-424f-acb0-038a3e238a4f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:16.291417 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7240901-4f6d-424f-acb0-038a3e238a4f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:16.292227 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7240901-4f6d-424f-acb0-038a3e238a4f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:16.292720 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7240901-4f6d-424f-acb0-038a3e238a4f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:16.293145 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7240901-4f6d-424f-acb0-038a3e238a4f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:16.426038 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c7240901-4f6d-424f-acb0-038a3e238a4f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:16.434102 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c7240901-4f6d-424f-acb0-038a3e238a4f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1260 microversion: 2.1 time: 0.155833 Oct 24 08:54:16.434604 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 189/367] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:16 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1260 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:16.474291 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-bc37672a-22dc-413e-b002-225a21f758bd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:16.474889 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-bc37672a-22dc-413e-b002-225a21f758bd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:16.482584 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc37672a-22dc-413e-b002-225a21f758bd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:16.483064 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc37672a-22dc-413e-b002-225a21f758bd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:16.483500 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc37672a-22dc-413e-b002-225a21f758bd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:16.484221 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc37672a-22dc-413e-b002-225a21f758bd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:16.484668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc37672a-22dc-413e-b002-225a21f758bd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:16.485084 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc37672a-22dc-413e-b002-225a21f758bd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:16.619323 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-bc37672a-22dc-413e-b002-225a21f758bd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:16.627990 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-bc37672a-22dc-413e-b002-225a21f758bd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:16.629463 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-bc37672a-22dc-413e-b002-225a21f758bd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:16.630624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-bc37672a-22dc-413e-b002-225a21f758bd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1723 microversion: 2.20 time: 0.160281 Oct 24 08:54:16.631181 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 179/368] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:16 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1723 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:17.457381 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-2311e864-603c-4a54-b5c1-7d323fec32c4 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:17.458058 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-2311e864-603c-4a54-b5c1-7d323fec32c4 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:17.465210 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2311e864-603c-4a54-b5c1-7d323fec32c4 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:17.465705 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2311e864-603c-4a54-b5c1-7d323fec32c4 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:17.466140 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2311e864-603c-4a54-b5c1-7d323fec32c4 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:17.466857 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2311e864-603c-4a54-b5c1-7d323fec32c4 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:17.467303 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2311e864-603c-4a54-b5c1-7d323fec32c4 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:17.467723 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2311e864-603c-4a54-b5c1-7d323fec32c4 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:17.610860 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-2311e864-603c-4a54-b5c1-7d323fec32c4 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:17.620269 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-2311e864-603c-4a54-b5c1-7d323fec32c4 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1260 microversion: 2.1 time: 0.166703 Oct 24 08:54:17.620783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 190/369] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:17 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1260 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:17.656277 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0afe84d8-c137-4b59-ba45-a14006be0efa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:17.656876 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-0afe84d8-c137-4b59-ba45-a14006be0efa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:17.663668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0afe84d8-c137-4b59-ba45-a14006be0efa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:17.664180 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0afe84d8-c137-4b59-ba45-a14006be0efa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:17.664628 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0afe84d8-c137-4b59-ba45-a14006be0efa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:17.665337 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0afe84d8-c137-4b59-ba45-a14006be0efa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:17.666536 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0afe84d8-c137-4b59-ba45-a14006be0efa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:17.666970 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0afe84d8-c137-4b59-ba45-a14006be0efa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:17.816542 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0afe84d8-c137-4b59-ba45-a14006be0efa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:17.824654 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0afe84d8-c137-4b59-ba45-a14006be0efa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:17.826095 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0afe84d8-c137-4b59-ba45-a14006be0efa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:17.827250 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-0afe84d8-c137-4b59-ba45-a14006be0efa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1723 microversion: 2.20 time: 0.175106 Oct 24 08:54:17.827803 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 180/370] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:17 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1723 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:18.643547 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d8e2df5f-143f-4b84-bdf7-8f8280c4f14d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:18.645433 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d8e2df5f-143f-4b84-bdf7-8f8280c4f14d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:18.650742 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d8e2df5f-143f-4b84-bdf7-8f8280c4f14d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:18.651326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d8e2df5f-143f-4b84-bdf7-8f8280c4f14d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:18.651768 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d8e2df5f-143f-4b84-bdf7-8f8280c4f14d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:18.652504 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d8e2df5f-143f-4b84-bdf7-8f8280c4f14d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:18.652968 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d8e2df5f-143f-4b84-bdf7-8f8280c4f14d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:18.653399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d8e2df5f-143f-4b84-bdf7-8f8280c4f14d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:18.791157 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d8e2df5f-143f-4b84-bdf7-8f8280c4f14d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:18.799386 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d8e2df5f-143f-4b84-bdf7-8f8280c4f14d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1260 microversion: 2.1 time: 0.159737 Oct 24 08:54:18.799962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 191/371] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:18 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1260 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:18.853903 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6daab74f-4b52-43ba-a16c-6f58e8f4ba4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:18.854519 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6daab74f-4b52-43ba-a16c-6f58e8f4ba4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:18.861316 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6daab74f-4b52-43ba-a16c-6f58e8f4ba4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:18.861829 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6daab74f-4b52-43ba-a16c-6f58e8f4ba4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:18.862272 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6daab74f-4b52-43ba-a16c-6f58e8f4ba4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:18.862980 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6daab74f-4b52-43ba-a16c-6f58e8f4ba4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:18.863441 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6daab74f-4b52-43ba-a16c-6f58e8f4ba4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:18.863867 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6daab74f-4b52-43ba-a16c-6f58e8f4ba4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:19.003622 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6daab74f-4b52-43ba-a16c-6f58e8f4ba4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:19.012883 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6daab74f-4b52-43ba-a16c-6f58e8f4ba4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:19.014320 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6daab74f-4b52-43ba-a16c-6f58e8f4ba4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:19.015558 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6daab74f-4b52-43ba-a16c-6f58e8f4ba4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1723 microversion: 2.20 time: 0.165646 Oct 24 08:54:19.016036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 181/372] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:18 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1723 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:19.823199 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-29267426-7732-4288-9d6a-7fc10aa364cc tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:19.823902 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-29267426-7732-4288-9d6a-7fc10aa364cc tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:19.830129 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-29267426-7732-4288-9d6a-7fc10aa364cc tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:19.830620 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-29267426-7732-4288-9d6a-7fc10aa364cc tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:19.831061 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-29267426-7732-4288-9d6a-7fc10aa364cc tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:19.831766 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-29267426-7732-4288-9d6a-7fc10aa364cc tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:19.832245 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-29267426-7732-4288-9d6a-7fc10aa364cc tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:19.832675 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-29267426-7732-4288-9d6a-7fc10aa364cc tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:19.981175 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-29267426-7732-4288-9d6a-7fc10aa364cc tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:19.989405 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-29267426-7732-4288-9d6a-7fc10aa364cc tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1260 microversion: 2.1 time: 0.170262 Oct 24 08:54:19.990257 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 192/373] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:19 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1260 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:20.042055 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6ea9689e-1a4c-4c55-af38-3dc84221237d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:20.042875 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6ea9689e-1a4c-4c55-af38-3dc84221237d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:20.049252 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6ea9689e-1a4c-4c55-af38-3dc84221237d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:20.049735 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6ea9689e-1a4c-4c55-af38-3dc84221237d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:20.050169 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6ea9689e-1a4c-4c55-af38-3dc84221237d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:20.050862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6ea9689e-1a4c-4c55-af38-3dc84221237d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:20.051313 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6ea9689e-1a4c-4c55-af38-3dc84221237d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:20.051738 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6ea9689e-1a4c-4c55-af38-3dc84221237d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:20.192760 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6ea9689e-1a4c-4c55-af38-3dc84221237d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:20.201687 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6ea9689e-1a4c-4c55-af38-3dc84221237d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:20.203157 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6ea9689e-1a4c-4c55-af38-3dc84221237d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:20.204516 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6ea9689e-1a4c-4c55-af38-3dc84221237d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1723 microversion: 2.20 time: 0.167393 Oct 24 08:54:20.205099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 182/374] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:20 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1723 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:21.014839 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a4ffa980-aeaf-4226-9a95-d06ac75aeca1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:21.015444 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a4ffa980-aeaf-4226-9a95-d06ac75aeca1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:21.017481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=2 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:21.018151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=6 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:21.024370 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4ffa980-aeaf-4226-9a95-d06ac75aeca1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:21.024921 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4ffa980-aeaf-4226-9a95-d06ac75aeca1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:21.026076 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4ffa980-aeaf-4226-9a95-d06ac75aeca1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:21.026811 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4ffa980-aeaf-4226-9a95-d06ac75aeca1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:21.027263 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4ffa980-aeaf-4226-9a95-d06ac75aeca1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:21.027683 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4ffa980-aeaf-4226-9a95-d06ac75aeca1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:21.169971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a4ffa980-aeaf-4226-9a95-d06ac75aeca1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:21.177997 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a4ffa980-aeaf-4226-9a95-d06ac75aeca1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1260 microversion: 2.1 time: 0.168390 Oct 24 08:54:21.178600 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 193/375] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:21 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1260 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:21.232096 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a68ee7a5-6ad5-4620-a39e-c1fd1521e4e7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:21.232786 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-a68ee7a5-6ad5-4620-a39e-c1fd1521e4e7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:21.234740 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=2 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:21.240883 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a68ee7a5-6ad5-4620-a39e-c1fd1521e4e7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:21.241374 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a68ee7a5-6ad5-4620-a39e-c1fd1521e4e7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:21.241810 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a68ee7a5-6ad5-4620-a39e-c1fd1521e4e7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:21.242520 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a68ee7a5-6ad5-4620-a39e-c1fd1521e4e7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:21.242975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a68ee7a5-6ad5-4620-a39e-c1fd1521e4e7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:21.243433 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a68ee7a5-6ad5-4620-a39e-c1fd1521e4e7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:21.382606 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-a68ee7a5-6ad5-4620-a39e-c1fd1521e4e7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:21.391234 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-a68ee7a5-6ad5-4620-a39e-c1fd1521e4e7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:21.392721 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-a68ee7a5-6ad5-4620-a39e-c1fd1521e4e7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:21.393892 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-a68ee7a5-6ad5-4620-a39e-c1fd1521e4e7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1723 microversion: 2.20 time: 0.166413 Oct 24 08:54:21.394442 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 183/376] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:21 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1723 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:54:22.202335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-884178b5-3f7e-4308-9ce0-df036548affa tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:22.202943 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-884178b5-3f7e-4308-9ce0-df036548affa tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:22.209249 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-884178b5-3f7e-4308-9ce0-df036548affa tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:22.209479 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-884178b5-3f7e-4308-9ce0-df036548affa tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:22.209901 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-884178b5-3f7e-4308-9ce0-df036548affa tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:22.210865 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-884178b5-3f7e-4308-9ce0-df036548affa tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:22.213139 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-884178b5-3f7e-4308-9ce0-df036548affa tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:22.213139 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-884178b5-3f7e-4308-9ce0-df036548affa tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:22.365287 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-884178b5-3f7e-4308-9ce0-df036548affa tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:22.374758 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-884178b5-3f7e-4308-9ce0-df036548affa tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1260 microversion: 2.1 time: 0.176108 Oct 24 08:54:22.375376 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 194/377] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:22 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1260 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:22.420511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b8e32249-a321-4693-9a0c-8f92d1045e2a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:22.421130 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b8e32249-a321-4693-9a0c-8f92d1045e2a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:22.427895 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b8e32249-a321-4693-9a0c-8f92d1045e2a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:22.428419 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b8e32249-a321-4693-9a0c-8f92d1045e2a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:22.428862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b8e32249-a321-4693-9a0c-8f92d1045e2a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:22.429708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b8e32249-a321-4693-9a0c-8f92d1045e2a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:22.430187 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b8e32249-a321-4693-9a0c-8f92d1045e2a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:22.430725 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b8e32249-a321-4693-9a0c-8f92d1045e2a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:22.575768 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b8e32249-a321-4693-9a0c-8f92d1045e2a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:22.583972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b8e32249-a321-4693-9a0c-8f92d1045e2a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:22.585467 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b8e32249-a321-4693-9a0c-8f92d1045e2a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:22.586600 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b8e32249-a321-4693-9a0c-8f92d1045e2a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1723 microversion: 2.20 time: 0.170693 Oct 24 08:54:22.587148 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 184/378] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:22 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1723 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:54:23.399554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d59fae54-a3e0-4c28-8b88-25e8985b9e7b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:23.399999 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d59fae54-a3e0-4c28-8b88-25e8985b9e7b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:23.406834 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d59fae54-a3e0-4c28-8b88-25e8985b9e7b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:23.407321 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d59fae54-a3e0-4c28-8b88-25e8985b9e7b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:23.407766 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d59fae54-a3e0-4c28-8b88-25e8985b9e7b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:23.408486 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d59fae54-a3e0-4c28-8b88-25e8985b9e7b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:23.408936 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d59fae54-a3e0-4c28-8b88-25e8985b9e7b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:23.409357 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d59fae54-a3e0-4c28-8b88-25e8985b9e7b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:23.547997 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d59fae54-a3e0-4c28-8b88-25e8985b9e7b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:23.555992 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d59fae54-a3e0-4c28-8b88-25e8985b9e7b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1260 microversion: 2.1 time: 0.160746 Oct 24 08:54:23.557253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 195/379] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:23 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1260 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:23.613418 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-cf0118cc-a46f-4c1d-bd6e-16ab3027753c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:23.614100 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-cf0118cc-a46f-4c1d-bd6e-16ab3027753c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:23.621976 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cf0118cc-a46f-4c1d-bd6e-16ab3027753c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:23.622465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cf0118cc-a46f-4c1d-bd6e-16ab3027753c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:23.622905 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cf0118cc-a46f-4c1d-bd6e-16ab3027753c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:23.623628 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cf0118cc-a46f-4c1d-bd6e-16ab3027753c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:23.624111 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cf0118cc-a46f-4c1d-bd6e-16ab3027753c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:23.624614 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cf0118cc-a46f-4c1d-bd6e-16ab3027753c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:23.764935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-cf0118cc-a46f-4c1d-bd6e-16ab3027753c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:23.773495 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-cf0118cc-a46f-4c1d-bd6e-16ab3027753c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:23.774972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-cf0118cc-a46f-4c1d-bd6e-16ab3027753c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:23.776141 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-cf0118cc-a46f-4c1d-bd6e-16ab3027753c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1723 microversion: 2.20 time: 0.166999 Oct 24 08:54:23.776719 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 185/380] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:23 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1723 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:24.581041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c5f3b371-eaa4-4308-bcc8-09189e14b8c8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:24.581614 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c5f3b371-eaa4-4308-bcc8-09189e14b8c8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:24.588971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c5f3b371-eaa4-4308-bcc8-09189e14b8c8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:24.589220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c5f3b371-eaa4-4308-bcc8-09189e14b8c8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:24.589515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c5f3b371-eaa4-4308-bcc8-09189e14b8c8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:24.590313 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c5f3b371-eaa4-4308-bcc8-09189e14b8c8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:24.590674 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c5f3b371-eaa4-4308-bcc8-09189e14b8c8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:24.594237 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c5f3b371-eaa4-4308-bcc8-09189e14b8c8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:24.783146 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c5f3b371-eaa4-4308-bcc8-09189e14b8c8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:24.798429 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c5f3b371-eaa4-4308-bcc8-09189e14b8c8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1260 microversion: 2.1 time: 0.220780 Oct 24 08:54:24.799524 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 196/381] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:24 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1260 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:24.807091 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7b847b36-8b87-4480-a42f-20c30b76e68d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:24.809012 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7b847b36-8b87-4480-a42f-20c30b76e68d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:24.819395 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7b847b36-8b87-4480-a42f-20c30b76e68d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:24.819931 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7b847b36-8b87-4480-a42f-20c30b76e68d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:24.820431 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7b847b36-8b87-4480-a42f-20c30b76e68d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:24.821231 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7b847b36-8b87-4480-a42f-20c30b76e68d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:24.823001 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7b847b36-8b87-4480-a42f-20c30b76e68d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:24.823820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7b847b36-8b87-4480-a42f-20c30b76e68d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:24.991852 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7b847b36-8b87-4480-a42f-20c30b76e68d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:25.007323 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7b847b36-8b87-4480-a42f-20c30b76e68d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:25.010791 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7b847b36-8b87-4480-a42f-20c30b76e68d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:25.013096 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7b847b36-8b87-4480-a42f-20c30b76e68d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1723 microversion: 2.20 time: 0.213899 Oct 24 08:54:25.014147 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 186/382] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:24 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1723 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:25.821818 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-430717cc-f8de-4d3d-8c0d-a33bb4e0c72c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:25.822662 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-430717cc-f8de-4d3d-8c0d-a33bb4e0c72c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:25.830191 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-430717cc-f8de-4d3d-8c0d-a33bb4e0c72c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:25.830678 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-430717cc-f8de-4d3d-8c0d-a33bb4e0c72c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:25.831121 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-430717cc-f8de-4d3d-8c0d-a33bb4e0c72c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:25.831835 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-430717cc-f8de-4d3d-8c0d-a33bb4e0c72c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:25.832317 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-430717cc-f8de-4d3d-8c0d-a33bb4e0c72c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:25.832748 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-430717cc-f8de-4d3d-8c0d-a33bb4e0c72c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:25.969950 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-430717cc-f8de-4d3d-8c0d-a33bb4e0c72c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:25.978812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-430717cc-f8de-4d3d-8c0d-a33bb4e0c72c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1260 microversion: 2.1 time: 0.160502 Oct 24 08:54:25.979497 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 197/383] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:25 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1260 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:26.057224 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-494a51e9-6b5c-4b6b-856b-3c7f24248562 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:26.058031 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-494a51e9-6b5c-4b6b-856b-3c7f24248562 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:26.065179 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-494a51e9-6b5c-4b6b-856b-3c7f24248562 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:26.065663 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-494a51e9-6b5c-4b6b-856b-3c7f24248562 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:26.066090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-494a51e9-6b5c-4b6b-856b-3c7f24248562 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:26.066813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-494a51e9-6b5c-4b6b-856b-3c7f24248562 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:26.067357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-494a51e9-6b5c-4b6b-856b-3c7f24248562 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:26.067783 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-494a51e9-6b5c-4b6b-856b-3c7f24248562 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:26.211991 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-494a51e9-6b5c-4b6b-856b-3c7f24248562 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:26.221767 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-494a51e9-6b5c-4b6b-856b-3c7f24248562 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:26.223200 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-494a51e9-6b5c-4b6b-856b-3c7f24248562 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:26.224387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-494a51e9-6b5c-4b6b-856b-3c7f24248562 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1723 microversion: 2.20 time: 0.172942 Oct 24 08:54:26.224951 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 187/384] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:26 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1723 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:26.420073 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-af5bd47c-dabc-4b2b-96a8-fd054c01771c req-7f962fea-8a58-4b17-bd98-277cacb0b6f4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "354e90c0-1ef5-43e0-8983-2067982a3f76", "name": "network-vif-plugged", "status": "completed", "tag": "4a9dc16c-b094-43ae-a495-1098e5c9f054"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:54:26.430741 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-af5bd47c-dabc-4b2b-96a8-fd054c01771c req-7f962fea-8a58-4b17-bd98-277cacb0b6f4 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:26.431402 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-af5bd47c-dabc-4b2b-96a8-fd054c01771c req-7f962fea-8a58-4b17-bd98-277cacb0b6f4 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:26.432064 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-af5bd47c-dabc-4b2b-96a8-fd054c01771c req-7f962fea-8a58-4b17-bd98-277cacb0b6f4 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:26.446433 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-af5bd47c-dabc-4b2b-96a8-fd054c01771c req-7f962fea-8a58-4b17-bd98-277cacb0b6f4 service nova] Creating event network-vif-plugged:4a9dc16c-b094-43ae-a495-1098e5c9f054 for instance 354e90c0-1ef5-43e0-8983-2067982a3f76 on hv-cinder-maste Oct 24 08:54:26.453032 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-af5bd47c-dabc-4b2b-96a8-fd054c01771c req-7f962fea-8a58-4b17-bd98-277cacb0b6f4 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037200 Oct 24 08:54:26.453875 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 198/385] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:54:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:54:27.017379 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-46746cd2-92d7-43f2-98ab-82d7e0ece683 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:27.017910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-46746cd2-92d7-43f2-98ab-82d7e0ece683 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:27.023980 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-46746cd2-92d7-43f2-98ab-82d7e0ece683 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:27.024498 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-46746cd2-92d7-43f2-98ab-82d7e0ece683 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:27.025249 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-46746cd2-92d7-43f2-98ab-82d7e0ece683 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:27.025980 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-46746cd2-92d7-43f2-98ab-82d7e0ece683 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:27.026461 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-46746cd2-92d7-43f2-98ab-82d7e0ece683 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:27.026873 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-46746cd2-92d7-43f2-98ab-82d7e0ece683 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:27.172250 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-46746cd2-92d7-43f2-98ab-82d7e0ece683 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:27.180204 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-46746cd2-92d7-43f2-98ab-82d7e0ece683 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1260 microversion: 2.1 time: 0.166811 Oct 24 08:54:27.181326 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 188/386] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:27 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1260 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:54:27.257030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1cb42d7f-7148-4ae2-b167-c9e1b9ab0fbb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:27.257752 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1cb42d7f-7148-4ae2-b167-c9e1b9ab0fbb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:27.264917 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1cb42d7f-7148-4ae2-b167-c9e1b9ab0fbb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:27.265415 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1cb42d7f-7148-4ae2-b167-c9e1b9ab0fbb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:27.265861 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1cb42d7f-7148-4ae2-b167-c9e1b9ab0fbb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:27.266581 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1cb42d7f-7148-4ae2-b167-c9e1b9ab0fbb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:27.267038 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1cb42d7f-7148-4ae2-b167-c9e1b9ab0fbb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:27.267521 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1cb42d7f-7148-4ae2-b167-c9e1b9ab0fbb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:27.406978 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1cb42d7f-7148-4ae2-b167-c9e1b9ab0fbb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:27.416011 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1cb42d7f-7148-4ae2-b167-c9e1b9ab0fbb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:27.417537 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1cb42d7f-7148-4ae2-b167-c9e1b9ab0fbb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:27.418723 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1cb42d7f-7148-4ae2-b167-c9e1b9ab0fbb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1723 microversion: 2.20 time: 0.166099 Oct 24 08:54:27.419287 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 199/387] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:27 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1723 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:28.203930 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5dcb166c-12df-456f-bb9b-93568d3b0c54 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:28.204559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-5dcb166c-12df-456f-bb9b-93568d3b0c54 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:28.206801 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 24 08:54:28.206941 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Mon Oct 24 08:54:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 (ip 10.222.0.39) !!! Oct 24 08:54:28.214440 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5dcb166c-12df-456f-bb9b-93568d3b0c54 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:28.214924 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5dcb166c-12df-456f-bb9b-93568d3b0c54 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:28.215358 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5dcb166c-12df-456f-bb9b-93568d3b0c54 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:28.216055 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5dcb166c-12df-456f-bb9b-93568d3b0c54 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:28.216532 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5dcb166c-12df-456f-bb9b-93568d3b0c54 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:28.216953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5dcb166c-12df-456f-bb9b-93568d3b0c54 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:28.356383 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5dcb166c-12df-456f-bb9b-93568d3b0c54 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:28.365217 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5dcb166c-12df-456f-bb9b-93568d3b0c54 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1260 microversion: 2.1 time: 0.164766 Oct 24 08:54:28.365808 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 189/388] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:28 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1260 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:28.444843 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-755a58ec-c057-43a9-b247-0ca1c493a985 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:28.445528 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-755a58ec-c057-43a9-b247-0ca1c493a985 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:28.452283 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-755a58ec-c057-43a9-b247-0ca1c493a985 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:28.453106 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-755a58ec-c057-43a9-b247-0ca1c493a985 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:28.453592 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-755a58ec-c057-43a9-b247-0ca1c493a985 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:28.454367 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-755a58ec-c057-43a9-b247-0ca1c493a985 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:28.454818 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-755a58ec-c057-43a9-b247-0ca1c493a985 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:28.455242 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-755a58ec-c057-43a9-b247-0ca1c493a985 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:28.592829 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-755a58ec-c057-43a9-b247-0ca1c493a985 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:28.596295 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:28.602034 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-755a58ec-c057-43a9-b247-0ca1c493a985 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:28.603512 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-755a58ec-c057-43a9-b247-0ca1c493a985 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:28.604725 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-755a58ec-c057-43a9-b247-0ca1c493a985 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1744 microversion: 2.20 time: 0.164330 Oct 24 08:54:28.605279 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 200/389] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:28 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1744 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:29.389895 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-d7686ccb-3d9f-4ea8-9435-59fd961b8d98 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:29.390593 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-d7686ccb-3d9f-4ea8-9435-59fd961b8d98 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:29.397520 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7686ccb-3d9f-4ea8-9435-59fd961b8d98 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:29.398006 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7686ccb-3d9f-4ea8-9435-59fd961b8d98 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:29.398444 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7686ccb-3d9f-4ea8-9435-59fd961b8d98 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:29.399141 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7686ccb-3d9f-4ea8-9435-59fd961b8d98 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:29.400643 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7686ccb-3d9f-4ea8-9435-59fd961b8d98 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:29.400881 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7686ccb-3d9f-4ea8-9435-59fd961b8d98 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:29.531189 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d7686ccb-3d9f-4ea8-9435-59fd961b8d98 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:29.538859 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-d7686ccb-3d9f-4ea8-9435-59fd961b8d98 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1445 microversion: 2.1 time: 0.153354 Oct 24 08:54:29.539421 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 190/390] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:29 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1445 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:54:29.561948 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b98d0a48-edba-4b95-9771-6f01c80185d0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:29.562558 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b98d0a48-edba-4b95-9771-6f01c80185d0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:29.569688 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b98d0a48-edba-4b95-9771-6f01c80185d0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:29.570174 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b98d0a48-edba-4b95-9771-6f01c80185d0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:29.570614 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b98d0a48-edba-4b95-9771-6f01c80185d0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:29.571307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b98d0a48-edba-4b95-9771-6f01c80185d0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:29.572365 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b98d0a48-edba-4b95-9771-6f01c80185d0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:29.572795 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b98d0a48-edba-4b95-9771-6f01c80185d0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:29.631168 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-faa026ae-9d53-4ce0-8a60-2aee8e97995e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:29.631948 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-faa026ae-9d53-4ce0-8a60-2aee8e97995e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:29.639474 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-faa026ae-9d53-4ce0-8a60-2aee8e97995e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:29.640022 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-faa026ae-9d53-4ce0-8a60-2aee8e97995e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:29.640603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-faa026ae-9d53-4ce0-8a60-2aee8e97995e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:29.641334 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-faa026ae-9d53-4ce0-8a60-2aee8e97995e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:29.641791 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-faa026ae-9d53-4ce0-8a60-2aee8e97995e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:29.642215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-faa026ae-9d53-4ce0-8a60-2aee8e97995e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:29.669844 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=100 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:29.756929 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-b98d0a48-edba-4b95-9771-6f01c80185d0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:29.765718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b98d0a48-edba-4b95-9771-6f01c80185d0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1445 microversion: 2.1 time: 0.207334 Oct 24 08:54:29.772251 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 201/391] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:29 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1445 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:29.804226 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-faa026ae-9d53-4ce0-8a60-2aee8e97995e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:29.813247 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-faa026ae-9d53-4ce0-8a60-2aee8e97995e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:29.814708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-faa026ae-9d53-4ce0-8a60-2aee8e97995e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:29.815870 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-faa026ae-9d53-4ce0-8a60-2aee8e97995e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1744 microversion: 2.20 time: 0.188687 Oct 24 08:54:29.816720 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 191/392] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:29 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1744 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:54:30.403404 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-de256757-d080-4dcb-95ce-82e1250728ad req-78a83024-c2e5-461d-9359-2e5221d6bd7e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "71e106ba-ad54-4e9c-8e42-9477c8db8cf6", "name": "network-vif-plugged", "status": "completed", "tag": "b8754c97-b769-418c-b461-714a5809d621"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:54:30.428196 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-de256757-d080-4dcb-95ce-82e1250728ad req-78a83024-c2e5-461d-9359-2e5221d6bd7e service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:30.428831 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-de256757-d080-4dcb-95ce-82e1250728ad req-78a83024-c2e5-461d-9359-2e5221d6bd7e service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:30.429332 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-de256757-d080-4dcb-95ce-82e1250728ad req-78a83024-c2e5-461d-9359-2e5221d6bd7e service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:30.442486 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-de256757-d080-4dcb-95ce-82e1250728ad req-78a83024-c2e5-461d-9359-2e5221d6bd7e service nova] Creating event network-vif-plugged:b8754c97-b769-418c-b461-714a5809d621 for instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 on hv-cinder-maste Oct 24 08:54:30.447974 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-de256757-d080-4dcb-95ce-82e1250728ad req-78a83024-c2e5-461d-9359-2e5221d6bd7e service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055564 Oct 24 08:54:30.448767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 202/393] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:54:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:54:30.843388 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ac33760a-6847-48dc-910b-10a26b62ac4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:30.844083 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ac33760a-6847-48dc-910b-10a26b62ac4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:30.850531 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ac33760a-6847-48dc-910b-10a26b62ac4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:30.851092 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ac33760a-6847-48dc-910b-10a26b62ac4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:30.851528 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ac33760a-6847-48dc-910b-10a26b62ac4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:30.852254 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ac33760a-6847-48dc-910b-10a26b62ac4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:30.852743 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ac33760a-6847-48dc-910b-10a26b62ac4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:30.853730 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ac33760a-6847-48dc-910b-10a26b62ac4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:30.998180 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ac33760a-6847-48dc-910b-10a26b62ac4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:31.006488 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ac33760a-6847-48dc-910b-10a26b62ac4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:31.007918 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ac33760a-6847-48dc-910b-10a26b62ac4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:31.009829 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ac33760a-6847-48dc-910b-10a26b62ac4f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1744 microversion: 2.20 time: 0.169618 Oct 24 08:54:31.010394 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 192/394] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:30 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1744 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:54:31.575952 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-0d9b039b-5f0e-4c09-8cec-9dc55781b0b7 req-b8506426-69ce-4f84-b12e-631c1a15f8e8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "354e90c0-1ef5-43e0-8983-2067982a3f76", "tag": "4a9dc16c-b094-43ae-a495-1098e5c9f054"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:54:31.585493 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-0d9b039b-5f0e-4c09-8cec-9dc55781b0b7 req-b8506426-69ce-4f84-b12e-631c1a15f8e8 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:31.585967 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-0d9b039b-5f0e-4c09-8cec-9dc55781b0b7 req-b8506426-69ce-4f84-b12e-631c1a15f8e8 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:31.586392 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-0d9b039b-5f0e-4c09-8cec-9dc55781b0b7 req-b8506426-69ce-4f84-b12e-631c1a15f8e8 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:31.597492 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-0d9b039b-5f0e-4c09-8cec-9dc55781b0b7 req-b8506426-69ce-4f84-b12e-631c1a15f8e8 service nova] Creating event network-changed:4a9dc16c-b094-43ae-a495-1098e5c9f054 for instance 354e90c0-1ef5-43e0-8983-2067982a3f76 on hv-cinder-maste Oct 24 08:54:31.602566 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-0d9b039b-5f0e-4c09-8cec-9dc55781b0b7 req-b8506426-69ce-4f84-b12e-631c1a15f8e8 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031456 Oct 24 08:54:31.603323 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 203/395] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:54:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:54:31.663821 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-18a13930-d3da-4f2e-bb15-57ea05a79b75 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:31.664565 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-18a13930-d3da-4f2e-bb15-57ea05a79b75 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:31.671992 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-18a13930-d3da-4f2e-bb15-57ea05a79b75 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:31.672856 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-18a13930-d3da-4f2e-bb15-57ea05a79b75 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:31.673483 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-18a13930-d3da-4f2e-bb15-57ea05a79b75 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:31.674224 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-18a13930-d3da-4f2e-bb15-57ea05a79b75 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:31.674839 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-18a13930-d3da-4f2e-bb15-57ea05a79b75 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:31.675304 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-18a13930-d3da-4f2e-bb15-57ea05a79b75 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:31.701512 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-18a13930-d3da-4f2e-bb15-57ea05a79b75 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1/ips" status: 200 len: 149 microversion: 2.1 time: 0.041482 Oct 24 08:54:31.702337 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 193/396] 10.222.0.39 () {58 vars in 1248 bytes} [Mon Oct 24 08:54:31 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1/ips => generated 149 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:54:32.035684 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8f50ee77-aa06-4556-a945-d63605d45959 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:32.036465 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8f50ee77-aa06-4556-a945-d63605d45959 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:32.046399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8f50ee77-aa06-4556-a945-d63605d45959 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:32.046897 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8f50ee77-aa06-4556-a945-d63605d45959 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:32.047342 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8f50ee77-aa06-4556-a945-d63605d45959 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:32.048056 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8f50ee77-aa06-4556-a945-d63605d45959 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:32.051072 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8f50ee77-aa06-4556-a945-d63605d45959 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:32.051524 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8f50ee77-aa06-4556-a945-d63605d45959 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:32.214849 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8f50ee77-aa06-4556-a945-d63605d45959 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:32.223464 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8f50ee77-aa06-4556-a945-d63605d45959 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:32.224984 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8f50ee77-aa06-4556-a945-d63605d45959 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:32.226203 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8f50ee77-aa06-4556-a945-d63605d45959 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1744 microversion: 2.20 time: 0.194798 Oct 24 08:54:32.226767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 204/397] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:32 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1744 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:33.253991 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-641860ba-a8b1-4afd-b57a-f0b3c5955d94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:33.254600 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-641860ba-a8b1-4afd-b57a-f0b3c5955d94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:33.261157 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-641860ba-a8b1-4afd-b57a-f0b3c5955d94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:33.261631 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-641860ba-a8b1-4afd-b57a-f0b3c5955d94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:33.262081 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-641860ba-a8b1-4afd-b57a-f0b3c5955d94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:33.262746 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-641860ba-a8b1-4afd-b57a-f0b3c5955d94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:33.263195 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-641860ba-a8b1-4afd-b57a-f0b3c5955d94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:33.263611 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-641860ba-a8b1-4afd-b57a-f0b3c5955d94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:33.395080 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-641860ba-a8b1-4afd-b57a-f0b3c5955d94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:33.403168 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-641860ba-a8b1-4afd-b57a-f0b3c5955d94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:33.404640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-641860ba-a8b1-4afd-b57a-f0b3c5955d94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:33.405822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-641860ba-a8b1-4afd-b57a-f0b3c5955d94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1744 microversion: 2.20 time: 0.155762 Oct 24 08:54:33.406379 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 194/398] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:33 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1744 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:34.195807 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7392f0c7-310a-4a2e-b934-356fdc1b0875 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c14733e6-7fc6-4b57-b829-55b2304a14e8"}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:54:34.199208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7392f0c7-310a-4a2e-b934-356fdc1b0875 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:34.205324 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7392f0c7-310a-4a2e-b934-356fdc1b0875 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:34.205801 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7392f0c7-310a-4a2e-b934-356fdc1b0875 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:34.206229 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7392f0c7-310a-4a2e-b934-356fdc1b0875 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:34.206921 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7392f0c7-310a-4a2e-b934-356fdc1b0875 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:34.207365 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7392f0c7-310a-4a2e-b934-356fdc1b0875 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:34.207848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7392f0c7-310a-4a2e-b934-356fdc1b0875 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:34.432371 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9b80f6cb-a195-4876-ae39-412c785006d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:34.433069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9b80f6cb-a195-4876-ae39-412c785006d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:34.439754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9b80f6cb-a195-4876-ae39-412c785006d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:34.440315 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9b80f6cb-a195-4876-ae39-412c785006d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:34.440750 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9b80f6cb-a195-4876-ae39-412c785006d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:34.441455 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9b80f6cb-a195-4876-ae39-412c785006d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:34.441905 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9b80f6cb-a195-4876-ae39-412c785006d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:34.442327 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9b80f6cb-a195-4876-ae39-412c785006d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:34.501036 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7392f0c7-310a-4a2e-b934-356fdc1b0875 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "POST /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.308977 Oct 24 08:54:34.507148 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 205/399] 10.222.0.39 () {60 vars in 1323 bytes} [Mon Oct 24 08:54:34 2022] POST /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1/os-volume_attachments => generated 194 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:54:34.583599 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9b80f6cb-a195-4876-ae39-412c785006d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:34.593103 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9b80f6cb-a195-4876-ae39-412c785006d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:34.594555 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9b80f6cb-a195-4876-ae39-412c785006d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:34.595722 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9b80f6cb-a195-4876-ae39-412c785006d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1744 microversion: 2.20 time: 0.167601 Oct 24 08:54:34.596329 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 195/400] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:34 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1744 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:54:35.623332 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e367ec8d-f65e-47d1-bd6f-a5aeffe1d611 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:35.624022 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e367ec8d-f65e-47d1-bd6f-a5aeffe1d611 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:35.631345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e367ec8d-f65e-47d1-bd6f-a5aeffe1d611 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:35.631830 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e367ec8d-f65e-47d1-bd6f-a5aeffe1d611 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:35.632302 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e367ec8d-f65e-47d1-bd6f-a5aeffe1d611 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:35.632999 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e367ec8d-f65e-47d1-bd6f-a5aeffe1d611 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:35.633460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e367ec8d-f65e-47d1-bd6f-a5aeffe1d611 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:35.634382 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e367ec8d-f65e-47d1-bd6f-a5aeffe1d611 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:35.765122 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e367ec8d-f65e-47d1-bd6f-a5aeffe1d611 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:35.773228 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e367ec8d-f65e-47d1-bd6f-a5aeffe1d611 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:35.775275 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e367ec8d-f65e-47d1-bd6f-a5aeffe1d611 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:35.776469 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e367ec8d-f65e-47d1-bd6f-a5aeffe1d611 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1744 microversion: 2.20 time: 0.157398 Oct 24 08:54:35.777028 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 206/401] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:35 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1744 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:36.802055 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5d018dee-bfab-4991-89f1-92d531614be2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:36.802657 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-5d018dee-bfab-4991-89f1-92d531614be2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:36.810313 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5d018dee-bfab-4991-89f1-92d531614be2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:36.811930 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5d018dee-bfab-4991-89f1-92d531614be2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:36.811930 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5d018dee-bfab-4991-89f1-92d531614be2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:36.812445 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5d018dee-bfab-4991-89f1-92d531614be2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:36.812445 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5d018dee-bfab-4991-89f1-92d531614be2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:36.812862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5d018dee-bfab-4991-89f1-92d531614be2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:36.988931 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5d018dee-bfab-4991-89f1-92d531614be2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:36.998960 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5d018dee-bfab-4991-89f1-92d531614be2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:37.000432 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5d018dee-bfab-4991-89f1-92d531614be2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:37.001597 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5d018dee-bfab-4991-89f1-92d531614be2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1739 microversion: 2.20 time: 0.203381 Oct 24 08:54:37.002146 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 196/402] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:36 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1739 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:38.030240 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c07b6fa6-2534-4c72-8703-8a0f5f55355e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:38.030948 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c07b6fa6-2534-4c72-8703-8a0f5f55355e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:38.038205 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c07b6fa6-2534-4c72-8703-8a0f5f55355e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:38.038902 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c07b6fa6-2534-4c72-8703-8a0f5f55355e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:38.039579 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c07b6fa6-2534-4c72-8703-8a0f5f55355e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:38.041134 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c07b6fa6-2534-4c72-8703-8a0f5f55355e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:38.041814 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c07b6fa6-2534-4c72-8703-8a0f5f55355e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:38.042526 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c07b6fa6-2534-4c72-8703-8a0f5f55355e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:38.195322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c07b6fa6-2534-4c72-8703-8a0f5f55355e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:38.204676 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c07b6fa6-2534-4c72-8703-8a0f5f55355e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:38.206128 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c07b6fa6-2534-4c72-8703-8a0f5f55355e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:38.207296 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c07b6fa6-2534-4c72-8703-8a0f5f55355e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1739 microversion: 2.20 time: 0.181103 Oct 24 08:54:38.207827 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 207/403] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:38 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1739 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:54:39.234833 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-496ce6c4-afda-43bb-bc89-ec48d26c1b6e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:39.235493 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-496ce6c4-afda-43bb-bc89-ec48d26c1b6e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:39.243150 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-496ce6c4-afda-43bb-bc89-ec48d26c1b6e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:39.244630 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-496ce6c4-afda-43bb-bc89-ec48d26c1b6e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:39.245680 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-496ce6c4-afda-43bb-bc89-ec48d26c1b6e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:39.246399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-496ce6c4-afda-43bb-bc89-ec48d26c1b6e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:39.246997 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-496ce6c4-afda-43bb-bc89-ec48d26c1b6e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:39.247493 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-496ce6c4-afda-43bb-bc89-ec48d26c1b6e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:39.403873 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-496ce6c4-afda-43bb-bc89-ec48d26c1b6e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:39.411440 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-496ce6c4-afda-43bb-bc89-ec48d26c1b6e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:39.412821 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-496ce6c4-afda-43bb-bc89-ec48d26c1b6e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:39.414184 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-496ce6c4-afda-43bb-bc89-ec48d26c1b6e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1739 microversion: 2.20 time: 0.183319 Oct 24 08:54:39.415435 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 197/404] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:39 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1739 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:40.441558 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a258084e-e36d-4bef-9bb2-8da78597fd5f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:40.442394 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a258084e-e36d-4bef-9bb2-8da78597fd5f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:40.444900 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=62,nova_api:INSERT=3 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:40.450460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a258084e-e36d-4bef-9bb2-8da78597fd5f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:40.450955 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a258084e-e36d-4bef-9bb2-8da78597fd5f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:40.451486 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a258084e-e36d-4bef-9bb2-8da78597fd5f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:40.452161 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a258084e-e36d-4bef-9bb2-8da78597fd5f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:40.452656 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a258084e-e36d-4bef-9bb2-8da78597fd5f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:40.453357 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a258084e-e36d-4bef-9bb2-8da78597fd5f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:40.608238 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a258084e-e36d-4bef-9bb2-8da78597fd5f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:40.623831 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a258084e-e36d-4bef-9bb2-8da78597fd5f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:40.625573 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a258084e-e36d-4bef-9bb2-8da78597fd5f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:40.627499 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a258084e-e36d-4bef-9bb2-8da78597fd5f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1739 microversion: 2.20 time: 0.189555 Oct 24 08:54:40.628663 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 208/405] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:40 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1739 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:41.655373 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-85f3f8a1-0a01-43d0-8a39-ebbcf3a64128 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:41.655981 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-85f3f8a1-0a01-43d0-8a39-ebbcf3a64128 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:41.658558 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=51 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:41.666775 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-85f3f8a1-0a01-43d0-8a39-ebbcf3a64128 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:41.667262 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-85f3f8a1-0a01-43d0-8a39-ebbcf3a64128 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:41.667718 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-85f3f8a1-0a01-43d0-8a39-ebbcf3a64128 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:41.674935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-85f3f8a1-0a01-43d0-8a39-ebbcf3a64128 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:41.675652 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-85f3f8a1-0a01-43d0-8a39-ebbcf3a64128 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:41.676346 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-85f3f8a1-0a01-43d0-8a39-ebbcf3a64128 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:41.857182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-85f3f8a1-0a01-43d0-8a39-ebbcf3a64128 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:41.866127 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-85f3f8a1-0a01-43d0-8a39-ebbcf3a64128 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:41.868548 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-85f3f8a1-0a01-43d0-8a39-ebbcf3a64128 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:41.869848 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-85f3f8a1-0a01-43d0-8a39-ebbcf3a64128 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1739 microversion: 2.20 time: 0.218473 Oct 24 08:54:41.885155 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 198/406] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:41 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1739 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:42.188509 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-0d539c32-eb1f-44f6-9589-dd6a423e8d5d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Action: 'create', calling method: .version_select at 0x7fdfedea2ee0>, body: {"keypair": {"name": "tempest-keypair-1885294353", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMydjTe3lvstWvrrbbynFaiQvhwV4jd6NS8GOwYngzbW03LPElfSKLQPi8dZJNqlxfuhLSr4icPqI/Hwt2M5t1U/s5vBbVvVAVawS70VokNosUOJ1pC3AVm21Jb+GCkaeQ=="}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:54:42.204480 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-0d539c32-eb1f-44f6-9589-dd6a423e8d5d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Getting quotas for project b34f10ec378a4622a8f0b0546e84da7f. Resources: {'key_pairs'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:54:42.208256 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-0d539c32-eb1f-44f6-9589-dd6a423e8d5d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Getting quotas for user ebdef1d69bb14837b9a0c28cbba332fc and project b34f10ec378a4622a8f0b0546e84da7f. Resources: {'key_pairs'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:54:42.262298 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-0d539c32-eb1f-44f6-9589-dd6a423e8d5d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.077676 Oct 24 08:54:42.263004 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 209/407] 10.222.0.39 () {60 vars in 1159 bytes} [Mon Oct 24 08:54:42 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:54:42.897345 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e007d8e5-8d5b-4c64-9288-f23120ce62c2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:42.898383 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e007d8e5-8d5b-4c64-9288-f23120ce62c2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:42.907191 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e007d8e5-8d5b-4c64-9288-f23120ce62c2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:42.907997 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e007d8e5-8d5b-4c64-9288-f23120ce62c2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:42.908957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e007d8e5-8d5b-4c64-9288-f23120ce62c2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:42.909967 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e007d8e5-8d5b-4c64-9288-f23120ce62c2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:42.910656 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e007d8e5-8d5b-4c64-9288-f23120ce62c2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:42.911292 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e007d8e5-8d5b-4c64-9288-f23120ce62c2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:43.092048 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e007d8e5-8d5b-4c64-9288-f23120ce62c2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:43.106289 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e007d8e5-8d5b-4c64-9288-f23120ce62c2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:43.107814 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e007d8e5-8d5b-4c64-9288-f23120ce62c2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:43.109186 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e007d8e5-8d5b-4c64-9288-f23120ce62c2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1739 microversion: 2.20 time: 0.215746 Oct 24 08:54:43.109843 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 199/408] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:42 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1739 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:43.983156 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1549365840", "imageRef": "bba331ba-faf5-4a56-81d9-6e84e9609665", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "151b561f-53f8-4019-8ed3-b08a9e67fe57"}], "security_groups": [{"name": "tempest-securitygroup--1912974456"}], "key_name": "tempest-keypair-1885294353", "user_data": "}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:54:44.098456 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.network.neutron [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] validate_networks() for [('151b561f-53f8-4019-8ed3-b08a9e67fe57', None, None, None, None, None)] {{(pid=130395) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 08:54:44.137644 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-20f75c1d-d4b6-4ec3-8796-ab48e6280efd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:44.138999 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-20f75c1d-d4b6-4ec3-8796-ab48e6280efd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:44.146885 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-20f75c1d-d4b6-4ec3-8796-ab48e6280efd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:44.147473 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-20f75c1d-d4b6-4ec3-8796-ab48e6280efd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:44.147994 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-20f75c1d-d4b6-4ec3-8796-ab48e6280efd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:44.148809 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-20f75c1d-d4b6-4ec3-8796-ab48e6280efd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:44.149878 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-20f75c1d-d4b6-4ec3-8796-ab48e6280efd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:44.150380 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-20f75c1d-d4b6-4ec3-8796-ab48e6280efd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:44.338840 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-20f75c1d-d4b6-4ec3-8796-ab48e6280efd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:44.348994 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-20f75c1d-d4b6-4ec3-8796-ab48e6280efd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:44.350705 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-20f75c1d-d4b6-4ec3-8796-ab48e6280efd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:44.353568 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-20f75c1d-d4b6-4ec3-8796-ab48e6280efd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1739 microversion: 2.20 time: 0.219874 Oct 24 08:54:44.369823 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 200/409] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:44 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1739 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:54:44.618566 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Flavor limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 08:54:44.618950 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Image limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 08:54:44.619655 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Flavor pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 08:54:44.620162 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Image pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 08:54:44.620657 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 08:54:44.630973 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:44.631679 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:44.632113 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:44.632884 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:44.633866 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:44.634466 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:44.654228 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Getting quotas for project b34f10ec378a4622a8f0b0546e84da7f. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:54:44.658233 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Getting quotas for user ebdef1d69bb14837b9a0c28cbba332fc and project b34f10ec378a4622a8f0b0546e84da7f. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:54:44.670832 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Going to run 1 instances... {{(pid=130395) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 08:54:44.673638 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bba331ba-faf5-4a56-81d9-6e84e9609665',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130395) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 08:54:44.728303 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b73a5f77-89e7-426e-944c-b186358deeaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.749606 Oct 24 08:54:44.729198 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 210/410] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 08:54:43 2022] POST /compute/v2.1/servers => generated 402 bytes in 752 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 08:54:44.740576 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-97dec5a8-c67a-4203-9fc1-0141568d81a3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:44.741185 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-97dec5a8-c67a-4203-9fc1-0141568d81a3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:44.825740 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-97dec5a8-c67a-4203-9fc1-0141568d81a3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:44.833305 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-97dec5a8-c67a-4203-9fc1-0141568d81a3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1242 microversion: 2.1 time: 0.096856 Oct 24 08:54:44.840213 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 201/411] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:44 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1242 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:45.420399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d27cded6-9fe0-4b42-beb0-3063219cc1c4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:45.420399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d27cded6-9fe0-4b42-beb0-3063219cc1c4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:45.462847 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d27cded6-9fe0-4b42-beb0-3063219cc1c4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:45.463793 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d27cded6-9fe0-4b42-beb0-3063219cc1c4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:45.464532 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d27cded6-9fe0-4b42-beb0-3063219cc1c4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:45.465508 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d27cded6-9fe0-4b42-beb0-3063219cc1c4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:45.466188 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d27cded6-9fe0-4b42-beb0-3063219cc1c4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:45.466850 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d27cded6-9fe0-4b42-beb0-3063219cc1c4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:45.618762 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d27cded6-9fe0-4b42-beb0-3063219cc1c4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:45.629080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d27cded6-9fe0-4b42-beb0-3063219cc1c4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:45.631384 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d27cded6-9fe0-4b42-beb0-3063219cc1c4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:45.632763 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d27cded6-9fe0-4b42-beb0-3063219cc1c4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1739 microversion: 2.20 time: 0.247476 Oct 24 08:54:45.633602 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 211/412] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:45 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1739 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:45.858710 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-69a019fc-824b-4e5d-a0ed-9b654afee97a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:45.859318 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-69a019fc-824b-4e5d-a0ed-9b654afee97a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:45.872538 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-69a019fc-824b-4e5d-a0ed-9b654afee97a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:45.873391 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-69a019fc-824b-4e5d-a0ed-9b654afee97a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:45.874143 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-69a019fc-824b-4e5d-a0ed-9b654afee97a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:45.875325 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-69a019fc-824b-4e5d-a0ed-9b654afee97a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:45.875905 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-69a019fc-824b-4e5d-a0ed-9b654afee97a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:45.876546 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-69a019fc-824b-4e5d-a0ed-9b654afee97a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:45.952951 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-69a019fc-824b-4e5d-a0ed-9b654afee97a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:45.966209 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-69a019fc-824b-4e5d-a0ed-9b654afee97a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1294 microversion: 2.1 time: 0.111224 Oct 24 08:54:45.967274 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 202/413] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:45 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1294 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:46.659641 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-aa987811-328d-442b-9d9a-c5cc930f8a50 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:46.660286 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-aa987811-328d-442b-9d9a-c5cc930f8a50 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:46.666877 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa987811-328d-442b-9d9a-c5cc930f8a50 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:46.667355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa987811-328d-442b-9d9a-c5cc930f8a50 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:46.667792 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa987811-328d-442b-9d9a-c5cc930f8a50 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:46.668519 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa987811-328d-442b-9d9a-c5cc930f8a50 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:46.668971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa987811-328d-442b-9d9a-c5cc930f8a50 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:46.669393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa987811-328d-442b-9d9a-c5cc930f8a50 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:46.799003 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-aa987811-328d-442b-9d9a-c5cc930f8a50 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:46.807925 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-aa987811-328d-442b-9d9a-c5cc930f8a50 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:46.809404 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-aa987811-328d-442b-9d9a-c5cc930f8a50 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:46.810557 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-aa987811-328d-442b-9d9a-c5cc930f8a50 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1739 microversion: 2.20 time: 0.154812 Oct 24 08:54:46.811116 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 212/414] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:46 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1739 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:46.990468 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-141199e9-73b4-4935-a2aa-e93ff1b0ed67 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:46.991117 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-141199e9-73b4-4935-a2aa-e93ff1b0ed67 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:46.998639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-141199e9-73b4-4935-a2aa-e93ff1b0ed67 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:46.999126 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-141199e9-73b4-4935-a2aa-e93ff1b0ed67 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:46.999568 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-141199e9-73b4-4935-a2aa-e93ff1b0ed67 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:47.000311 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-141199e9-73b4-4935-a2aa-e93ff1b0ed67 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:47.000774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-141199e9-73b4-4935-a2aa-e93ff1b0ed67 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:47.001202 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-141199e9-73b4-4935-a2aa-e93ff1b0ed67 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:47.062546 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-141199e9-73b4-4935-a2aa-e93ff1b0ed67 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:47.071027 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-141199e9-73b4-4935-a2aa-e93ff1b0ed67 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1300 microversion: 2.1 time: 0.084663 Oct 24 08:54:47.071828 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 203/415] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:46 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1300 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:47.836026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a131aa83-50b1-4db7-9e2e-85e3211c6e19 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:47.837383 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a131aa83-50b1-4db7-9e2e-85e3211c6e19 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:47.844176 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a131aa83-50b1-4db7-9e2e-85e3211c6e19 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:47.844911 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a131aa83-50b1-4db7-9e2e-85e3211c6e19 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:47.845378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a131aa83-50b1-4db7-9e2e-85e3211c6e19 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:47.846259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a131aa83-50b1-4db7-9e2e-85e3211c6e19 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:47.846742 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a131aa83-50b1-4db7-9e2e-85e3211c6e19 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:47.847159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a131aa83-50b1-4db7-9e2e-85e3211c6e19 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:48.072942 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a131aa83-50b1-4db7-9e2e-85e3211c6e19 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:48.081179 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a131aa83-50b1-4db7-9e2e-85e3211c6e19 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:48.082649 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a131aa83-50b1-4db7-9e2e-85e3211c6e19 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:48.084012 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a131aa83-50b1-4db7-9e2e-85e3211c6e19 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1721 microversion: 2.20 time: 0.251960 Oct 24 08:54:48.095401 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9c3e78b3-eb35-477f-b878-17f4143d0ce7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:48.097527 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9c3e78b3-eb35-477f-b878-17f4143d0ce7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:48.098067 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 213/416] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:47 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1721 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:48.108788 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c3e78b3-eb35-477f-b878-17f4143d0ce7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:48.109432 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c3e78b3-eb35-477f-b878-17f4143d0ce7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:48.109971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c3e78b3-eb35-477f-b878-17f4143d0ce7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:48.110825 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c3e78b3-eb35-477f-b878-17f4143d0ce7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:48.111495 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c3e78b3-eb35-477f-b878-17f4143d0ce7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:48.111984 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c3e78b3-eb35-477f-b878-17f4143d0ce7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:48.323029 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9c3e78b3-eb35-477f-b878-17f4143d0ce7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:48.331161 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9c3e78b3-eb35-477f-b878-17f4143d0ce7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.241003 Oct 24 08:54:48.331926 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 204/417] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:48 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:48.423757 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-cbe50e87-7324-4d54-89ff-3efc4bfa95c7 req-97e071bd-4a03-498a-8b13-9285fb97b2b3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8f61c54c-9a6d-445b-8a49-056657208b85", "tag": "e9bb3baa-6f12-4847-a485-980d16c4f05b"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:54:48.433796 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-cbe50e87-7324-4d54-89ff-3efc4bfa95c7 req-97e071bd-4a03-498a-8b13-9285fb97b2b3 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:48.434274 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-cbe50e87-7324-4d54-89ff-3efc4bfa95c7 req-97e071bd-4a03-498a-8b13-9285fb97b2b3 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:48.434707 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-cbe50e87-7324-4d54-89ff-3efc4bfa95c7 req-97e071bd-4a03-498a-8b13-9285fb97b2b3 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:48.445453 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-cbe50e87-7324-4d54-89ff-3efc4bfa95c7 req-97e071bd-4a03-498a-8b13-9285fb97b2b3 service nova] Creating event network-changed:e9bb3baa-6f12-4847-a485-980d16c4f05b for instance 8f61c54c-9a6d-445b-8a49-056657208b85 on hv-cinder-maste Oct 24 08:54:48.450447 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-cbe50e87-7324-4d54-89ff-3efc4bfa95c7 req-97e071bd-4a03-498a-8b13-9285fb97b2b3 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030351 Oct 24 08:54:48.451029 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 214/418] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:54:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:54:49.110213 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7229bbe2-0ae4-49ef-b062-3c4282791208 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:49.110820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7229bbe2-0ae4-49ef-b062-3c4282791208 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:49.117698 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7229bbe2-0ae4-49ef-b062-3c4282791208 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:49.118496 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7229bbe2-0ae4-49ef-b062-3c4282791208 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:49.118607 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7229bbe2-0ae4-49ef-b062-3c4282791208 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:49.119405 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7229bbe2-0ae4-49ef-b062-3c4282791208 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:49.119761 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7229bbe2-0ae4-49ef-b062-3c4282791208 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:49.120213 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7229bbe2-0ae4-49ef-b062-3c4282791208 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:49.255380 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7229bbe2-0ae4-49ef-b062-3c4282791208 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:49.264341 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7229bbe2-0ae4-49ef-b062-3c4282791208 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:49.265780 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7229bbe2-0ae4-49ef-b062-3c4282791208 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:49.266954 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7229bbe2-0ae4-49ef-b062-3c4282791208 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1721 microversion: 2.20 time: 0.160691 Oct 24 08:54:49.267494 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 205/419] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:49 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1721 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:49.355032 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-9c738813-76e0-43f4-85d5-1762cf50c481 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:49.355736 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-9c738813-76e0-43f4-85d5-1762cf50c481 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:49.361914 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c738813-76e0-43f4-85d5-1762cf50c481 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:49.362400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c738813-76e0-43f4-85d5-1762cf50c481 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:49.362846 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c738813-76e0-43f4-85d5-1762cf50c481 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:49.363624 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c738813-76e0-43f4-85d5-1762cf50c481 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:49.364034 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c738813-76e0-43f4-85d5-1762cf50c481 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:49.364489 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c738813-76e0-43f4-85d5-1762cf50c481 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:49.494111 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-9c738813-76e0-43f4-85d5-1762cf50c481 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:49.501906 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-9c738813-76e0-43f4-85d5-1762cf50c481 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.151731 Oct 24 08:54:49.502476 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 215/420] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:49 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:50.293100 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e42f95db-4a4e-48da-9681-767eaa1dcc36 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:50.293710 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e42f95db-4a4e-48da-9681-767eaa1dcc36 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:50.300369 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e42f95db-4a4e-48da-9681-767eaa1dcc36 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:50.301175 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e42f95db-4a4e-48da-9681-767eaa1dcc36 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:50.301657 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e42f95db-4a4e-48da-9681-767eaa1dcc36 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:50.302368 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e42f95db-4a4e-48da-9681-767eaa1dcc36 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:50.302829 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e42f95db-4a4e-48da-9681-767eaa1dcc36 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:50.303258 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e42f95db-4a4e-48da-9681-767eaa1dcc36 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:50.462964 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-cf221938-adce-4608-be2e-3980d219c028 req-020eebc4-4131-4eb2-a9ab-369ba91fa249 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "71e106ba-ad54-4e9c-8e42-9477c8db8cf6", "name": "network-vif-unplugged", "status": "completed", "tag": "b8754c97-b769-418c-b461-714a5809d621"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:54:50.480817 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e42f95db-4a4e-48da-9681-767eaa1dcc36 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:50.490918 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e42f95db-4a4e-48da-9681-767eaa1dcc36 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:50.492414 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e42f95db-4a4e-48da-9681-767eaa1dcc36 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:50.493951 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e42f95db-4a4e-48da-9681-767eaa1dcc36 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1721 microversion: 2.20 time: 0.204822 Oct 24 08:54:50.494516 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 206/421] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:50 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1721 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:50.496847 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-cf221938-adce-4608-be2e-3980d219c028 req-020eebc4-4131-4eb2-a9ab-369ba91fa249 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:50.497405 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-cf221938-adce-4608-be2e-3980d219c028 req-020eebc4-4131-4eb2-a9ab-369ba91fa249 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:50.497908 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-cf221938-adce-4608-be2e-3980d219c028 req-020eebc4-4131-4eb2-a9ab-369ba91fa249 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:50.510958 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-cf221938-adce-4608-be2e-3980d219c028 req-020eebc4-4131-4eb2-a9ab-369ba91fa249 service nova] Creating event network-vif-unplugged:b8754c97-b769-418c-b461-714a5809d621 for instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 on hv-cinder-maste Oct 24 08:54:50.517313 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-cf221938-adce-4608-be2e-3980d219c028 req-020eebc4-4131-4eb2-a9ab-369ba91fa249 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.059263 Oct 24 08:54:50.519922 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 216/422] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:54:50 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:54:50.527095 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-53e89e46-757f-46fa-a2e0-17b5abe414b2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:50.527986 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-53e89e46-757f-46fa-a2e0-17b5abe414b2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:50.535018 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-53e89e46-757f-46fa-a2e0-17b5abe414b2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:50.535520 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-53e89e46-757f-46fa-a2e0-17b5abe414b2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:50.535961 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-53e89e46-757f-46fa-a2e0-17b5abe414b2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:50.536710 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-53e89e46-757f-46fa-a2e0-17b5abe414b2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:50.537168 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-53e89e46-757f-46fa-a2e0-17b5abe414b2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:50.537592 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-53e89e46-757f-46fa-a2e0-17b5abe414b2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:50.676895 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-53e89e46-757f-46fa-a2e0-17b5abe414b2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:50.686383 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-53e89e46-757f-46fa-a2e0-17b5abe414b2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.165368 Oct 24 08:54:50.687174 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 207/423] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:50 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:51.524098 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a695ef32-d96e-49b0-978a-0e01190c3233 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:51.524853 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a695ef32-d96e-49b0-978a-0e01190c3233 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:51.530996 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a695ef32-d96e-49b0-978a-0e01190c3233 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:51.531482 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a695ef32-d96e-49b0-978a-0e01190c3233 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:51.531922 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a695ef32-d96e-49b0-978a-0e01190c3233 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:51.532669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a695ef32-d96e-49b0-978a-0e01190c3233 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:51.533152 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a695ef32-d96e-49b0-978a-0e01190c3233 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:51.533601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a695ef32-d96e-49b0-978a-0e01190c3233 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:51.662981 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a695ef32-d96e-49b0-978a-0e01190c3233 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:51.671931 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a695ef32-d96e-49b0-978a-0e01190c3233 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:51.673398 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a695ef32-d96e-49b0-978a-0e01190c3233 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:51.674554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a695ef32-d96e-49b0-978a-0e01190c3233 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1664 microversion: 2.20 time: 0.155181 Oct 24 08:54:51.675107 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 217/424] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:51 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1664 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:51.700015 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f5558255-9cef-4a59-bad8-4297274036b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6bbd93af-7386-4dd2-a515-4a6221cff28e"}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:54:51.702437 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f5558255-9cef-4a59-bad8-4297274036b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:51.708659 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-fb57efc7-9e2b-42ee-acd3-3ec0359c2cbd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:51.708860 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f5558255-9cef-4a59-bad8-4297274036b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:51.709224 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f5558255-9cef-4a59-bad8-4297274036b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:51.709340 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-fb57efc7-9e2b-42ee-acd3-3ec0359c2cbd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:51.709667 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f5558255-9cef-4a59-bad8-4297274036b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:51.710354 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f5558255-9cef-4a59-bad8-4297274036b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:51.710798 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f5558255-9cef-4a59-bad8-4297274036b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:51.711882 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f5558255-9cef-4a59-bad8-4297274036b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:51.716081 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fb57efc7-9e2b-42ee-acd3-3ec0359c2cbd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:51.716605 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fb57efc7-9e2b-42ee-acd3-3ec0359c2cbd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:51.717046 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fb57efc7-9e2b-42ee-acd3-3ec0359c2cbd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:51.717750 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fb57efc7-9e2b-42ee-acd3-3ec0359c2cbd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:51.718207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fb57efc7-9e2b-42ee-acd3-3ec0359c2cbd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:51.718636 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fb57efc7-9e2b-42ee-acd3-3ec0359c2cbd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:51.859144 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-fb57efc7-9e2b-42ee-acd3-3ec0359c2cbd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:51.867009 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-fb57efc7-9e2b-42ee-acd3-3ec0359c2cbd tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.162072 Oct 24 08:54:51.867615 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 218/425] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:51 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:54:51.889503 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=98,nova_cell1:INSERT=1,nova_cell1:UPDATE=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:52.012602 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f5558255-9cef-4a59-bad8-4297274036b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "POST /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.317320 Oct 24 08:54:52.013204 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 208/426] 10.222.0.39 () {62 vars in 1364 bytes} [Mon Oct 24 08:54:51 2022] POST /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/os-volume_attachments => generated 188 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:54:52.143161 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e3a348be-130b-46b4-8d66-b67fcefe1101 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:54:52.143753 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e3a348be-130b-46b4-8d66-b67fcefe1101 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:52.149805 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e3a348be-130b-46b4-8d66-b67fcefe1101 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:52.150289 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e3a348be-130b-46b4-8d66-b67fcefe1101 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:52.150731 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e3a348be-130b-46b4-8d66-b67fcefe1101 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:52.151414 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e3a348be-130b-46b4-8d66-b67fcefe1101 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:52.151870 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e3a348be-130b-46b4-8d66-b67fcefe1101 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:52.152317 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e3a348be-130b-46b4-8d66-b67fcefe1101 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:52.195336 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e3a348be-130b-46b4-8d66-b67fcefe1101 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=130395) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 24 08:54:52.240042 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:52.264829 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e3a348be-130b-46b4-8d66-b67fcefe1101 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "POST /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/action" status: 202 len: 0 microversion: 2.20 time: 0.125438 Oct 24 08:54:52.265585 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 219/427] 10.222.0.39 () {62 vars in 1319 bytes} [Mon Oct 24 08:54:52 2022] POST /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/action => generated 0 bytes in 127 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 24 08:54:52.273858 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-285acc7d-a8db-4e4e-bde7-70cf8c920aeb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:52.274470 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-285acc7d-a8db-4e4e-bde7-70cf8c920aeb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:52.281059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-285acc7d-a8db-4e4e-bde7-70cf8c920aeb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:52.281549 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-285acc7d-a8db-4e4e-bde7-70cf8c920aeb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:52.281990 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-285acc7d-a8db-4e4e-bde7-70cf8c920aeb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:52.282691 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-285acc7d-a8db-4e4e-bde7-70cf8c920aeb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:52.283146 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-285acc7d-a8db-4e4e-bde7-70cf8c920aeb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:52.283574 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-285acc7d-a8db-4e4e-bde7-70cf8c920aeb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:52.421164 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-285acc7d-a8db-4e4e-bde7-70cf8c920aeb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:52.430042 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-285acc7d-a8db-4e4e-bde7-70cf8c920aeb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:52.432047 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-285acc7d-a8db-4e4e-bde7-70cf8c920aeb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:52.433236 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-285acc7d-a8db-4e4e-bde7-70cf8c920aeb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1750 microversion: 2.20 time: 0.163414 Oct 24 08:54:52.434075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 209/428] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:52 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1750 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:54:52.721817 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-76c51d41-bbbd-423a-9cf3-71e53e46f81c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:52.722397 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-76c51d41-bbbd-423a-9cf3-71e53e46f81c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:52.729021 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-76c51d41-bbbd-423a-9cf3-71e53e46f81c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:52.729511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-76c51d41-bbbd-423a-9cf3-71e53e46f81c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:52.729949 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-76c51d41-bbbd-423a-9cf3-71e53e46f81c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:52.730653 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-76c51d41-bbbd-423a-9cf3-71e53e46f81c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:52.731110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-76c51d41-bbbd-423a-9cf3-71e53e46f81c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:52.731534 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-76c51d41-bbbd-423a-9cf3-71e53e46f81c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:52.756559 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-76c51d41-bbbd-423a-9cf3-71e53e46f81c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/ips" status: 200 len: 145 microversion: 2.1 time: 0.039015 Oct 24 08:54:52.757135 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 220/429] 10.222.0.39 () {58 vars in 1248 bytes} [Mon Oct 24 08:54:52 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/ips => generated 145 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:54:52.889875 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-71c2ebdf-1714-4bf9-9982-e7c98d6ab1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:52.890494 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-71c2ebdf-1714-4bf9-9982-e7c98d6ab1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:52.896871 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-71c2ebdf-1714-4bf9-9982-e7c98d6ab1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:52.897358 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-71c2ebdf-1714-4bf9-9982-e7c98d6ab1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:52.897791 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-71c2ebdf-1714-4bf9-9982-e7c98d6ab1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:52.898482 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-71c2ebdf-1714-4bf9-9982-e7c98d6ab1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:52.899005 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-71c2ebdf-1714-4bf9-9982-e7c98d6ab1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:52.900125 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-71c2ebdf-1714-4bf9-9982-e7c98d6ab1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:53.045138 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-71c2ebdf-1714-4bf9-9982-e7c98d6ab1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:53.053353 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-71c2ebdf-1714-4bf9-9982-e7c98d6ab1c6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.167317 Oct 24 08:54:53.053992 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 210/430] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:52 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:53.459152 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ccf32e55-cc21-46ab-a6d1-29cee36bbecb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:53.459750 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ccf32e55-cc21-46ab-a6d1-29cee36bbecb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:53.467385 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ccf32e55-cc21-46ab-a6d1-29cee36bbecb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:53.467865 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ccf32e55-cc21-46ab-a6d1-29cee36bbecb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:53.468344 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ccf32e55-cc21-46ab-a6d1-29cee36bbecb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:53.469037 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ccf32e55-cc21-46ab-a6d1-29cee36bbecb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:53.469479 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ccf32e55-cc21-46ab-a6d1-29cee36bbecb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:53.469897 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ccf32e55-cc21-46ab-a6d1-29cee36bbecb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:53.600324 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ccf32e55-cc21-46ab-a6d1-29cee36bbecb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:53.609833 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ccf32e55-cc21-46ab-a6d1-29cee36bbecb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:53.611310 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ccf32e55-cc21-46ab-a6d1-29cee36bbecb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:53.612496 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ccf32e55-cc21-46ab-a6d1-29cee36bbecb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1750 microversion: 2.20 time: 0.157310 Oct 24 08:54:53.613263 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 221/431] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:53 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1750 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:54.078114 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0f778fcf-8699-46d2-b9af-22d10540a6c1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:54.078733 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-0f778fcf-8699-46d2-b9af-22d10540a6c1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:54.085124 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0f778fcf-8699-46d2-b9af-22d10540a6c1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:54.085612 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0f778fcf-8699-46d2-b9af-22d10540a6c1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:54.086052 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0f778fcf-8699-46d2-b9af-22d10540a6c1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:54.087109 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0f778fcf-8699-46d2-b9af-22d10540a6c1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:54.087522 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0f778fcf-8699-46d2-b9af-22d10540a6c1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:54.087981 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0f778fcf-8699-46d2-b9af-22d10540a6c1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:54.236839 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0f778fcf-8699-46d2-b9af-22d10540a6c1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:54.245693 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-0f778fcf-8699-46d2-b9af-22d10540a6c1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.171349 Oct 24 08:54:54.246194 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 211/432] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:54 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:54:54.639380 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-0778ca5c-fe6b-441a-b45d-5e39e6a8dee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:54.640044 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-0778ca5c-fe6b-441a-b45d-5e39e6a8dee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:54.643946 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=2 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:54.648006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0778ca5c-fe6b-441a-b45d-5e39e6a8dee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:54.648522 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0778ca5c-fe6b-441a-b45d-5e39e6a8dee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:54.650102 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0778ca5c-fe6b-441a-b45d-5e39e6a8dee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:54.650820 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0778ca5c-fe6b-441a-b45d-5e39e6a8dee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:54.651277 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0778ca5c-fe6b-441a-b45d-5e39e6a8dee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:54.651694 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0778ca5c-fe6b-441a-b45d-5e39e6a8dee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:54.668041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=10 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:54.788831 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0778ca5c-fe6b-441a-b45d-5e39e6a8dee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:54.798826 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0778ca5c-fe6b-441a-b45d-5e39e6a8dee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:54.800413 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0778ca5c-fe6b-441a-b45d-5e39e6a8dee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:54.801935 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-0778ca5c-fe6b-441a-b45d-5e39e6a8dee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1752 microversion: 2.20 time: 0.166481 Oct 24 08:54:54.802495 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 222/433] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:54 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1752 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:55.034246 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e878d4be-e11c-43b4-a199-71d40c07abf6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "44952977-e924-4e63-a390-c688647bdaba"}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:54:55.036769 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e878d4be-e11c-43b4-a199-71d40c07abf6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:55.038596 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=2 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:55.043919 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e878d4be-e11c-43b4-a199-71d40c07abf6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:55.044459 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e878d4be-e11c-43b4-a199-71d40c07abf6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:55.044898 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e878d4be-e11c-43b4-a199-71d40c07abf6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:55.045608 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e878d4be-e11c-43b4-a199-71d40c07abf6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:55.046063 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e878d4be-e11c-43b4-a199-71d40c07abf6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:55.046486 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e878d4be-e11c-43b4-a199-71d40c07abf6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:55.269578 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-cc1291dc-4fed-4a18-8e49-7d0885b359f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:55.270478 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-cc1291dc-4fed-4a18-8e49-7d0885b359f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:55.279390 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc1291dc-4fed-4a18-8e49-7d0885b359f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:55.280326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc1291dc-4fed-4a18-8e49-7d0885b359f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:55.280930 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc1291dc-4fed-4a18-8e49-7d0885b359f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:55.282008 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc1291dc-4fed-4a18-8e49-7d0885b359f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:55.282781 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc1291dc-4fed-4a18-8e49-7d0885b359f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:55.283215 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc1291dc-4fed-4a18-8e49-7d0885b359f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:55.391323 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e878d4be-e11c-43b4-a199-71d40c07abf6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "POST /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.360828 Oct 24 08:54:55.391943 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 212/434] 10.222.0.39 () {60 vars in 1323 bytes} [Mon Oct 24 08:54:55 2022] POST /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/os-volume_attachments => generated 194 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:54:55.437746 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cc1291dc-4fed-4a18-8e49-7d0885b359f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:55.448415 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-cc1291dc-4fed-4a18-8e49-7d0885b359f1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.181629 Oct 24 08:54:55.449348 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 223/435] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:55 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:54:55.829733 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2a930584-3000-4d10-8cb3-5b3fc035ff54 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:55.830367 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2a930584-3000-4d10-8cb3-5b3fc035ff54 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:55.837693 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2a930584-3000-4d10-8cb3-5b3fc035ff54 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:55.838174 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2a930584-3000-4d10-8cb3-5b3fc035ff54 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:55.838617 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2a930584-3000-4d10-8cb3-5b3fc035ff54 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:55.839311 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2a930584-3000-4d10-8cb3-5b3fc035ff54 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:55.839764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2a930584-3000-4d10-8cb3-5b3fc035ff54 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:55.840236 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2a930584-3000-4d10-8cb3-5b3fc035ff54 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:55.971871 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2a930584-3000-4d10-8cb3-5b3fc035ff54 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:55.981753 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2a930584-3000-4d10-8cb3-5b3fc035ff54 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:55.983194 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2a930584-3000-4d10-8cb3-5b3fc035ff54 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:55.984451 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2a930584-3000-4d10-8cb3-5b3fc035ff54 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.158965 Oct 24 08:54:55.985050 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 213/436] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:55 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:56.472427 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ef551e0c-4f4f-4105-bbb3-e0307ec53230 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:56.473037 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ef551e0c-4f4f-4105-bbb3-e0307ec53230 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:56.482323 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef551e0c-4f4f-4105-bbb3-e0307ec53230 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:56.482323 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef551e0c-4f4f-4105-bbb3-e0307ec53230 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:56.482323 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef551e0c-4f4f-4105-bbb3-e0307ec53230 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:56.483202 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef551e0c-4f4f-4105-bbb3-e0307ec53230 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:56.483202 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef551e0c-4f4f-4105-bbb3-e0307ec53230 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:56.483412 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef551e0c-4f4f-4105-bbb3-e0307ec53230 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:56.637663 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ef551e0c-4f4f-4105-bbb3-e0307ec53230 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:56.645853 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ef551e0c-4f4f-4105-bbb3-e0307ec53230 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.177304 Oct 24 08:54:56.646431 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 224/437] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:56 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:56.693171 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-be7786e0-f713-43eb-91b7-86327fc41932 req-9a59a8fa-460a-466c-8316-24d0053faa29 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8f61c54c-9a6d-445b-8a49-056657208b85", "name": "network-vif-plugged", "status": "completed", "tag": "e9bb3baa-6f12-4847-a485-980d16c4f05b"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:54:56.703533 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-be7786e0-f713-43eb-91b7-86327fc41932 req-9a59a8fa-460a-466c-8316-24d0053faa29 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:56.704181 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-be7786e0-f713-43eb-91b7-86327fc41932 req-9a59a8fa-460a-466c-8316-24d0053faa29 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:56.704623 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-be7786e0-f713-43eb-91b7-86327fc41932 req-9a59a8fa-460a-466c-8316-24d0053faa29 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:56.718096 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-be7786e0-f713-43eb-91b7-86327fc41932 req-9a59a8fa-460a-466c-8316-24d0053faa29 service nova] Creating event network-vif-plugged:e9bb3baa-6f12-4847-a485-980d16c4f05b for instance 8f61c54c-9a6d-445b-8a49-056657208b85 on hv-cinder-maste Oct 24 08:54:56.724225 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-be7786e0-f713-43eb-91b7-86327fc41932 req-9a59a8fa-460a-466c-8316-24d0053faa29 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034737 Oct 24 08:54:56.725091 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 214/438] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:54:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:54:57.011154 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-0d10db92-f941-4f79-8f8c-9e6266a91673 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:57.011769 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-0d10db92-f941-4f79-8f8c-9e6266a91673 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:57.017888 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0d10db92-f941-4f79-8f8c-9e6266a91673 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:57.018402 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0d10db92-f941-4f79-8f8c-9e6266a91673 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:57.018889 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0d10db92-f941-4f79-8f8c-9e6266a91673 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:57.019585 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0d10db92-f941-4f79-8f8c-9e6266a91673 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:57.020037 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0d10db92-f941-4f79-8f8c-9e6266a91673 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:57.020512 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0d10db92-f941-4f79-8f8c-9e6266a91673 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:57.154734 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0d10db92-f941-4f79-8f8c-9e6266a91673 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:57.163467 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0d10db92-f941-4f79-8f8c-9e6266a91673 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:57.165695 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0d10db92-f941-4f79-8f8c-9e6266a91673 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:57.166997 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-0d10db92-f941-4f79-8f8c-9e6266a91673 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.159989 Oct 24 08:54:57.167621 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 225/439] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:57 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:57.670752 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c9635599-db75-4203-b54f-6703de16cc88 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:57.671483 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c9635599-db75-4203-b54f-6703de16cc88 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:57.677601 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c9635599-db75-4203-b54f-6703de16cc88 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:57.678088 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c9635599-db75-4203-b54f-6703de16cc88 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:57.678526 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c9635599-db75-4203-b54f-6703de16cc88 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:57.679210 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c9635599-db75-4203-b54f-6703de16cc88 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:57.679662 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c9635599-db75-4203-b54f-6703de16cc88 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:57.680613 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c9635599-db75-4203-b54f-6703de16cc88 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:57.810919 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c9635599-db75-4203-b54f-6703de16cc88 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:57.819452 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c9635599-db75-4203-b54f-6703de16cc88 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.153216 Oct 24 08:54:57.820025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 215/440] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:57 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:58.192458 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-176e05b6-1248-49e1-ab2c-49ccb128f1c6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:58.193048 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-176e05b6-1248-49e1-ab2c-49ccb128f1c6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:58.200587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-176e05b6-1248-49e1-ab2c-49ccb128f1c6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:58.201065 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-176e05b6-1248-49e1-ab2c-49ccb128f1c6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:58.201503 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-176e05b6-1248-49e1-ab2c-49ccb128f1c6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:58.202186 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-176e05b6-1248-49e1-ab2c-49ccb128f1c6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:58.202630 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-176e05b6-1248-49e1-ab2c-49ccb128f1c6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:58.203049 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-176e05b6-1248-49e1-ab2c-49ccb128f1c6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:58.338979 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-176e05b6-1248-49e1-ab2c-49ccb128f1c6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:58.347649 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-176e05b6-1248-49e1-ab2c-49ccb128f1c6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:58.349132 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-176e05b6-1248-49e1-ab2c-49ccb128f1c6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:58.350302 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-176e05b6-1248-49e1-ab2c-49ccb128f1c6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.161682 Oct 24 08:54:58.350862 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 226/441] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:58 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:54:58.842819 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2ecfe6e3-d2b1-4a33-8282-c71b9b566e10 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:58.843510 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2ecfe6e3-d2b1-4a33-8282-c71b9b566e10 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:58.850177 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2ecfe6e3-d2b1-4a33-8282-c71b9b566e10 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:58.850673 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2ecfe6e3-d2b1-4a33-8282-c71b9b566e10 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:58.851112 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2ecfe6e3-d2b1-4a33-8282-c71b9b566e10 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:58.852362 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2ecfe6e3-d2b1-4a33-8282-c71b9b566e10 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:58.852816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2ecfe6e3-d2b1-4a33-8282-c71b9b566e10 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:58.853246 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2ecfe6e3-d2b1-4a33-8282-c71b9b566e10 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:58.993078 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2ecfe6e3-d2b1-4a33-8282-c71b9b566e10 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:59.000860 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2ecfe6e3-d2b1-4a33-8282-c71b9b566e10 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.161703 Oct 24 08:54:59.001267 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 216/442] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:54:58 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:54:59.377763 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-9c45bc6f-c192-49e1-9462-11657576bda0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:54:59.378583 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-9c45bc6f-c192-49e1-9462-11657576bda0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:54:59.386083 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c45bc6f-c192-49e1-9462-11657576bda0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:59.386581 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c45bc6f-c192-49e1-9462-11657576bda0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:59.387009 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c45bc6f-c192-49e1-9462-11657576bda0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:59.387712 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c45bc6f-c192-49e1-9462-11657576bda0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:54:59.388176 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c45bc6f-c192-49e1-9462-11657576bda0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:54:59.388611 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9c45bc6f-c192-49e1-9462-11657576bda0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:54:59.530807 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-9c45bc6f-c192-49e1-9462-11657576bda0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:59.540916 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-9c45bc6f-c192-49e1-9462-11657576bda0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:59.542349 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-9c45bc6f-c192-49e1-9462-11657576bda0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:54:59.543539 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-9c45bc6f-c192-49e1-9462-11657576bda0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.169874 Oct 24 08:54:59.544678 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 227/443] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:54:59 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:00.025331 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8dec1dc2-ad0e-4185-9f56-4c8daec5f071 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:00.025944 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8dec1dc2-ad0e-4185-9f56-4c8daec5f071 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:00.032471 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8dec1dc2-ad0e-4185-9f56-4c8daec5f071 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:00.032959 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8dec1dc2-ad0e-4185-9f56-4c8daec5f071 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:00.033401 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8dec1dc2-ad0e-4185-9f56-4c8daec5f071 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:00.034132 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8dec1dc2-ad0e-4185-9f56-4c8daec5f071 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:00.034588 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8dec1dc2-ad0e-4185-9f56-4c8daec5f071 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:00.035015 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8dec1dc2-ad0e-4185-9f56-4c8daec5f071 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:00.185699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8dec1dc2-ad0e-4185-9f56-4c8daec5f071 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:00.193425 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8dec1dc2-ad0e-4185-9f56-4c8daec5f071 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.172871 Oct 24 08:55:00.193991 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 217/444] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:00 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:00.572682 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-92d9622e-b23b-496c-9361-7657aea6348f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:00.573688 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-92d9622e-b23b-496c-9361-7657aea6348f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:00.580875 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-92d9622e-b23b-496c-9361-7657aea6348f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:00.581455 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-92d9622e-b23b-496c-9361-7657aea6348f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:00.581951 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-92d9622e-b23b-496c-9361-7657aea6348f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:00.582724 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-92d9622e-b23b-496c-9361-7657aea6348f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:00.583253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-92d9622e-b23b-496c-9361-7657aea6348f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:00.583697 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-92d9622e-b23b-496c-9361-7657aea6348f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:00.726568 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-92d9622e-b23b-496c-9361-7657aea6348f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:00.738292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-92d9622e-b23b-496c-9361-7657aea6348f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:00.740254 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-92d9622e-b23b-496c-9361-7657aea6348f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:00.741496 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-92d9622e-b23b-496c-9361-7657aea6348f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.174611 Oct 24 08:55:00.742797 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 228/445] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:00 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:01.216331 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-1b3cf5a3-d1c7-48b7-929f-26be849cbbfe tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:01.217051 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-1b3cf5a3-d1c7-48b7-929f-26be849cbbfe tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:01.223845 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1b3cf5a3-d1c7-48b7-929f-26be849cbbfe tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:01.224310 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1b3cf5a3-d1c7-48b7-929f-26be849cbbfe tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:01.224841 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1b3cf5a3-d1c7-48b7-929f-26be849cbbfe tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:01.225620 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1b3cf5a3-d1c7-48b7-929f-26be849cbbfe tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:01.226145 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1b3cf5a3-d1c7-48b7-929f-26be849cbbfe tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:01.227175 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1b3cf5a3-d1c7-48b7-929f-26be849cbbfe tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:01.378401 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-1b3cf5a3-d1c7-48b7-929f-26be849cbbfe tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:01.387306 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-1b3cf5a3-d1c7-48b7-929f-26be849cbbfe tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.174903 Oct 24 08:55:01.387889 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 218/446] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:01 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:01.768705 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a3855eda-6354-4fec-968e-7a666cfe4c94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:01.769338 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a3855eda-6354-4fec-968e-7a666cfe4c94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:01.777125 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a3855eda-6354-4fec-968e-7a666cfe4c94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:01.777605 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a3855eda-6354-4fec-968e-7a666cfe4c94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:01.778039 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a3855eda-6354-4fec-968e-7a666cfe4c94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:01.778731 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a3855eda-6354-4fec-968e-7a666cfe4c94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:01.779175 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a3855eda-6354-4fec-968e-7a666cfe4c94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:01.779616 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a3855eda-6354-4fec-968e-7a666cfe4c94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:01.917223 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a3855eda-6354-4fec-968e-7a666cfe4c94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:01.935765 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a3855eda-6354-4fec-968e-7a666cfe4c94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:01.937324 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a3855eda-6354-4fec-968e-7a666cfe4c94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:01.938510 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a3855eda-6354-4fec-968e-7a666cfe4c94 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.173680 Oct 24 08:55:01.939069 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 229/447] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:01 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:02.411461 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-12a76ff5-bbf5-4b4e-a30d-ab3243fff281 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:02.412090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-12a76ff5-bbf5-4b4e-a30d-ab3243fff281 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:02.419357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-12a76ff5-bbf5-4b4e-a30d-ab3243fff281 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:02.420188 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-12a76ff5-bbf5-4b4e-a30d-ab3243fff281 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:02.420848 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-12a76ff5-bbf5-4b4e-a30d-ab3243fff281 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:02.421756 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-12a76ff5-bbf5-4b4e-a30d-ab3243fff281 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:02.422549 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-12a76ff5-bbf5-4b4e-a30d-ab3243fff281 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:02.423165 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-12a76ff5-bbf5-4b4e-a30d-ab3243fff281 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:02.573472 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-12a76ff5-bbf5-4b4e-a30d-ab3243fff281 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:02.581532 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-12a76ff5-bbf5-4b4e-a30d-ab3243fff281 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.174691 Oct 24 08:55:02.582116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 219/448] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:02 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:02.965073 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f1f110db-1faa-4967-a749-2bdc13a2baa2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:02.965805 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f1f110db-1faa-4967-a749-2bdc13a2baa2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:02.973203 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f1f110db-1faa-4967-a749-2bdc13a2baa2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:02.973692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f1f110db-1faa-4967-a749-2bdc13a2baa2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:02.974916 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f1f110db-1faa-4967-a749-2bdc13a2baa2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:02.974916 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f1f110db-1faa-4967-a749-2bdc13a2baa2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:02.975805 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f1f110db-1faa-4967-a749-2bdc13a2baa2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:02.975805 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f1f110db-1faa-4967-a749-2bdc13a2baa2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:03.115794 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f1f110db-1faa-4967-a749-2bdc13a2baa2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:03.126222 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f1f110db-1faa-4967-a749-2bdc13a2baa2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:03.127697 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f1f110db-1faa-4967-a749-2bdc13a2baa2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:03.129194 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f1f110db-1faa-4967-a749-2bdc13a2baa2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.167953 Oct 24 08:55:03.130127 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 230/449] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:02 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:03.606761 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6bf01a31-6435-405d-a314-6001218a4171 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:03.607465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6bf01a31-6435-405d-a314-6001218a4171 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:03.614082 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6bf01a31-6435-405d-a314-6001218a4171 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:03.614747 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6bf01a31-6435-405d-a314-6001218a4171 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:03.615187 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6bf01a31-6435-405d-a314-6001218a4171 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:03.615897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6bf01a31-6435-405d-a314-6001218a4171 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:03.616378 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6bf01a31-6435-405d-a314-6001218a4171 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:03.616803 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6bf01a31-6435-405d-a314-6001218a4171 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:03.762980 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6bf01a31-6435-405d-a314-6001218a4171 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:03.771087 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6bf01a31-6435-405d-a314-6001218a4171 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.168270 Oct 24 08:55:03.771675 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 220/450] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:03 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:04.157013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-5c8aa089-0882-4bed-bec5-c573f3e1e43a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:04.158115 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-5c8aa089-0882-4bed-bec5-c573f3e1e43a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:04.165487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5c8aa089-0882-4bed-bec5-c573f3e1e43a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:04.165868 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5c8aa089-0882-4bed-bec5-c573f3e1e43a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:04.166312 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5c8aa089-0882-4bed-bec5-c573f3e1e43a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:04.167029 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5c8aa089-0882-4bed-bec5-c573f3e1e43a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:04.167488 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5c8aa089-0882-4bed-bec5-c573f3e1e43a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:04.167908 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5c8aa089-0882-4bed-bec5-c573f3e1e43a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:04.311194 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-5c8aa089-0882-4bed-bec5-c573f3e1e43a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:04.321851 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-5c8aa089-0882-4bed-bec5-c573f3e1e43a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:04.323543 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-5c8aa089-0882-4bed-bec5-c573f3e1e43a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:04.325089 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-5c8aa089-0882-4bed-bec5-c573f3e1e43a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.171836 Oct 24 08:55:04.325735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 231/451] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:04 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:04.797110 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ee450a6a-cefe-481d-b6c6-6845c9049c6d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:04.797830 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ee450a6a-cefe-481d-b6c6-6845c9049c6d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:04.804341 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee450a6a-cefe-481d-b6c6-6845c9049c6d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:04.805411 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee450a6a-cefe-481d-b6c6-6845c9049c6d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:04.805855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee450a6a-cefe-481d-b6c6-6845c9049c6d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:04.806576 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee450a6a-cefe-481d-b6c6-6845c9049c6d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:04.807008 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee450a6a-cefe-481d-b6c6-6845c9049c6d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:04.807430 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ee450a6a-cefe-481d-b6c6-6845c9049c6d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:04.954207 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ee450a6a-cefe-481d-b6c6-6845c9049c6d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:04.962705 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ee450a6a-cefe-481d-b6c6-6845c9049c6d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.169467 Oct 24 08:55:04.963320 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 221/452] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:04 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:55:05.351058 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-0aff1a4c-d6e9-4451-8cea-09768284522c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:05.352323 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-0aff1a4c-d6e9-4451-8cea-09768284522c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:05.359423 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0aff1a4c-d6e9-4451-8cea-09768284522c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:05.359899 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0aff1a4c-d6e9-4451-8cea-09768284522c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:05.360355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0aff1a4c-d6e9-4451-8cea-09768284522c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:05.361055 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0aff1a4c-d6e9-4451-8cea-09768284522c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:05.361499 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0aff1a4c-d6e9-4451-8cea-09768284522c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:05.361916 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0aff1a4c-d6e9-4451-8cea-09768284522c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:05.509154 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0aff1a4c-d6e9-4451-8cea-09768284522c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:05.517993 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0aff1a4c-d6e9-4451-8cea-09768284522c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:05.519430 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0aff1a4c-d6e9-4451-8cea-09768284522c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:05.520615 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-0aff1a4c-d6e9-4451-8cea-09768284522c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.173547 Oct 24 08:55:05.521518 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 232/453] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:05 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:05.989377 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7fb610f0-504d-416a-9e00-08f5453a784d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:05.990290 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7fb610f0-504d-416a-9e00-08f5453a784d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:05.997408 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb610f0-504d-416a-9e00-08f5453a784d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:05.997894 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb610f0-504d-416a-9e00-08f5453a784d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:05.998343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb610f0-504d-416a-9e00-08f5453a784d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:05.999044 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb610f0-504d-416a-9e00-08f5453a784d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:05.999498 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb610f0-504d-416a-9e00-08f5453a784d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:05.999922 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb610f0-504d-416a-9e00-08f5453a784d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:06.146939 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7fb610f0-504d-416a-9e00-08f5453a784d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:06.151099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=92,nova_cell1:INSERT=3,nova_cell1:UPDATE=5 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:06.156922 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7fb610f0-504d-416a-9e00-08f5453a784d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.171422 Oct 24 08:55:06.157851 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 222/454] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:05 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:06.546321 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-77e6a93a-7551-4125-ac6c-9392bba5622d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:06.546957 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-77e6a93a-7551-4125-ac6c-9392bba5622d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:06.553678 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-77e6a93a-7551-4125-ac6c-9392bba5622d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:06.554170 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-77e6a93a-7551-4125-ac6c-9392bba5622d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:06.554978 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-77e6a93a-7551-4125-ac6c-9392bba5622d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:06.555985 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-77e6a93a-7551-4125-ac6c-9392bba5622d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:06.556502 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-77e6a93a-7551-4125-ac6c-9392bba5622d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:06.556968 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-77e6a93a-7551-4125-ac6c-9392bba5622d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:06.702650 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-77e6a93a-7551-4125-ac6c-9392bba5622d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:06.712856 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-77e6a93a-7551-4125-ac6c-9392bba5622d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:06.714347 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-77e6a93a-7551-4125-ac6c-9392bba5622d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:06.715857 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-77e6a93a-7551-4125-ac6c-9392bba5622d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.173459 Oct 24 08:55:06.716468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 233/455] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:06 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:07.182798 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-98495c8e-5359-4721-95a4-0fb12f447ca9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:07.183608 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-98495c8e-5359-4721-95a4-0fb12f447ca9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:07.191384 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-98495c8e-5359-4721-95a4-0fb12f447ca9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:07.191897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-98495c8e-5359-4721-95a4-0fb12f447ca9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:07.192408 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-98495c8e-5359-4721-95a4-0fb12f447ca9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:07.193203 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-98495c8e-5359-4721-95a4-0fb12f447ca9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:07.194072 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-98495c8e-5359-4721-95a4-0fb12f447ca9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:07.194514 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-98495c8e-5359-4721-95a4-0fb12f447ca9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:07.337914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-98495c8e-5359-4721-95a4-0fb12f447ca9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:07.346772 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-98495c8e-5359-4721-95a4-0fb12f447ca9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.168872 Oct 24 08:55:07.347516 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 223/456] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:07 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:07.743215 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-576c4eda-4303-4bd1-9c02-77aa508dceca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:07.743812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-576c4eda-4303-4bd1-9c02-77aa508dceca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:07.750148 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-576c4eda-4303-4bd1-9c02-77aa508dceca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:07.750629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-576c4eda-4303-4bd1-9c02-77aa508dceca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:07.751063 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-576c4eda-4303-4bd1-9c02-77aa508dceca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:07.751753 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-576c4eda-4303-4bd1-9c02-77aa508dceca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:07.752257 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-576c4eda-4303-4bd1-9c02-77aa508dceca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:07.752684 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-576c4eda-4303-4bd1-9c02-77aa508dceca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:07.928482 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-576c4eda-4303-4bd1-9c02-77aa508dceca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:07.938282 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-576c4eda-4303-4bd1-9c02-77aa508dceca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:07.939954 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-576c4eda-4303-4bd1-9c02-77aa508dceca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:07.941315 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-576c4eda-4303-4bd1-9c02-77aa508dceca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.202318 Oct 24 08:55:07.942037 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 234/457] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:07 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:08.373252 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6a3d6e7e-d866-488d-9099-0fa1f9093593 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:08.373889 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6a3d6e7e-d866-488d-9099-0fa1f9093593 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:08.383533 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a3d6e7e-d866-488d-9099-0fa1f9093593 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:08.384461 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a3d6e7e-d866-488d-9099-0fa1f9093593 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:08.384953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a3d6e7e-d866-488d-9099-0fa1f9093593 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:08.385669 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a3d6e7e-d866-488d-9099-0fa1f9093593 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:08.386483 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a3d6e7e-d866-488d-9099-0fa1f9093593 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:08.387211 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a3d6e7e-d866-488d-9099-0fa1f9093593 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:08.544212 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6a3d6e7e-d866-488d-9099-0fa1f9093593 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:08.557284 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6a3d6e7e-d866-488d-9099-0fa1f9093593 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.188931 Oct 24 08:55:08.558027 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 224/458] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:08 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:08.968465 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-cdb43e8c-abcd-4a57-8261-e391117c13ad tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:08.969067 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-cdb43e8c-abcd-4a57-8261-e391117c13ad tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:08.976008 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cdb43e8c-abcd-4a57-8261-e391117c13ad tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:08.977921 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cdb43e8c-abcd-4a57-8261-e391117c13ad tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:08.978372 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cdb43e8c-abcd-4a57-8261-e391117c13ad tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:08.979118 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cdb43e8c-abcd-4a57-8261-e391117c13ad tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:08.979938 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cdb43e8c-abcd-4a57-8261-e391117c13ad tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:08.980542 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cdb43e8c-abcd-4a57-8261-e391117c13ad tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:09.121934 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cdb43e8c-abcd-4a57-8261-e391117c13ad tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:09.130699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cdb43e8c-abcd-4a57-8261-e391117c13ad tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:09.132380 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cdb43e8c-abcd-4a57-8261-e391117c13ad tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:09.133537 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-cdb43e8c-abcd-4a57-8261-e391117c13ad tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.169000 Oct 24 08:55:09.134158 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 235/459] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:08 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:09.580651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-79c3326f-19d3-4525-91f0-caee57c43b17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:09.581203 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-79c3326f-19d3-4525-91f0-caee57c43b17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:09.588323 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-79c3326f-19d3-4525-91f0-caee57c43b17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:09.588808 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-79c3326f-19d3-4525-91f0-caee57c43b17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:09.589245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-79c3326f-19d3-4525-91f0-caee57c43b17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:09.589943 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-79c3326f-19d3-4525-91f0-caee57c43b17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:09.590395 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-79c3326f-19d3-4525-91f0-caee57c43b17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:09.590855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-79c3326f-19d3-4525-91f0-caee57c43b17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:09.728326 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-79c3326f-19d3-4525-91f0-caee57c43b17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:09.735893 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-79c3326f-19d3-4525-91f0-caee57c43b17 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.159171 Oct 24 08:55:09.736483 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 225/460] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:09 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:10.160384 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-4d4eb6b5-f720-4711-bf36-037eedd2d570 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:10.160991 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-4d4eb6b5-f720-4711-bf36-037eedd2d570 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:10.167478 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d4eb6b5-f720-4711-bf36-037eedd2d570 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:10.167956 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d4eb6b5-f720-4711-bf36-037eedd2d570 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:10.168422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d4eb6b5-f720-4711-bf36-037eedd2d570 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:10.169116 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d4eb6b5-f720-4711-bf36-037eedd2d570 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:10.169566 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d4eb6b5-f720-4711-bf36-037eedd2d570 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:10.169988 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d4eb6b5-f720-4711-bf36-037eedd2d570 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:10.189540 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:10.303200 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-4d4eb6b5-f720-4711-bf36-037eedd2d570 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:10.311735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-4d4eb6b5-f720-4711-bf36-037eedd2d570 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:10.313196 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-4d4eb6b5-f720-4711-bf36-037eedd2d570 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:10.314362 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-4d4eb6b5-f720-4711-bf36-037eedd2d570 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.157916 Oct 24 08:55:10.314907 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 236/461] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:10 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:10.760525 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7ab2a0a7-acc2-4db6-9f53-2e2ad756e772 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:10.761120 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7ab2a0a7-acc2-4db6-9f53-2e2ad756e772 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:10.767539 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7ab2a0a7-acc2-4db6-9f53-2e2ad756e772 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:10.768062 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7ab2a0a7-acc2-4db6-9f53-2e2ad756e772 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:10.768499 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7ab2a0a7-acc2-4db6-9f53-2e2ad756e772 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:10.769195 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7ab2a0a7-acc2-4db6-9f53-2e2ad756e772 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:10.769650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7ab2a0a7-acc2-4db6-9f53-2e2ad756e772 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:10.770081 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7ab2a0a7-acc2-4db6-9f53-2e2ad756e772 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:10.896788 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7ab2a0a7-acc2-4db6-9f53-2e2ad756e772 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:10.905209 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7ab2a0a7-acc2-4db6-9f53-2e2ad756e772 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1368 microversion: 2.1 time: 0.149390 Oct 24 08:55:10.905783 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 226/462] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:10 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1368 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:11.340004 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-565d16b5-df2c-4c82-9cef-b1c9c0fccf53 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:11.340641 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-565d16b5-df2c-4c82-9cef-b1c9c0fccf53 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:11.342957 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=84,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:11.348381 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-565d16b5-df2c-4c82-9cef-b1c9c0fccf53 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:11.348862 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-565d16b5-df2c-4c82-9cef-b1c9c0fccf53 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:11.349300 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-565d16b5-df2c-4c82-9cef-b1c9c0fccf53 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:11.349996 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-565d16b5-df2c-4c82-9cef-b1c9c0fccf53 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:11.350444 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-565d16b5-df2c-4c82-9cef-b1c9c0fccf53 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:11.350869 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-565d16b5-df2c-4c82-9cef-b1c9c0fccf53 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:11.488163 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-565d16b5-df2c-4c82-9cef-b1c9c0fccf53 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:11.497941 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-565d16b5-df2c-4c82-9cef-b1c9c0fccf53 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:11.499380 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-565d16b5-df2c-4c82-9cef-b1c9c0fccf53 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:11.500578 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-565d16b5-df2c-4c82-9cef-b1c9c0fccf53 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.164692 Oct 24 08:55:11.501144 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 237/463] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:11 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:11.927632 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6bab8740-f1ce-493f-83bd-161c9ae6025a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:11.928237 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6bab8740-f1ce-493f-83bd-161c9ae6025a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:11.930698 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=60 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:11.935884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6bab8740-f1ce-493f-83bd-161c9ae6025a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:11.936391 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6bab8740-f1ce-493f-83bd-161c9ae6025a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:11.936826 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6bab8740-f1ce-493f-83bd-161c9ae6025a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:11.937513 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6bab8740-f1ce-493f-83bd-161c9ae6025a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:11.937957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6bab8740-f1ce-493f-83bd-161c9ae6025a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:11.938375 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6bab8740-f1ce-493f-83bd-161c9ae6025a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:12.066736 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6bab8740-f1ce-493f-83bd-161c9ae6025a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:12.075079 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6bab8740-f1ce-493f-83bd-161c9ae6025a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1559 microversion: 2.1 time: 0.151257 Oct 24 08:55:12.075654 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 227/464] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:11 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:12.097719 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-bd43f75c-ea4a-4a91-ae2f-5cd9de5f6f70 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:12.098206 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-bd43f75c-ea4a-4a91-ae2f-5cd9de5f6f70 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:12.105176 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bd43f75c-ea4a-4a91-ae2f-5cd9de5f6f70 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:12.105649 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bd43f75c-ea4a-4a91-ae2f-5cd9de5f6f70 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:12.106074 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bd43f75c-ea4a-4a91-ae2f-5cd9de5f6f70 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:12.106758 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bd43f75c-ea4a-4a91-ae2f-5cd9de5f6f70 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:12.107202 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bd43f75c-ea4a-4a91-ae2f-5cd9de5f6f70 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:12.107616 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bd43f75c-ea4a-4a91-ae2f-5cd9de5f6f70 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:12.189661 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-bd43f75c-ea4a-4a91-ae2f-5cd9de5f6f70 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.095749 Oct 24 08:55:12.190275 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 238/465] 10.222.0.39 () {58 vars in 1275 bytes} [Mon Oct 24 08:55:12 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85/os-interface => generated 285 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:55:12.526597 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f7df1649-14bc-42d6-ae9b-8f5a389e15f7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:12.527212 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f7df1649-14bc-42d6-ae9b-8f5a389e15f7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:12.534545 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7df1649-14bc-42d6-ae9b-8f5a389e15f7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:12.535052 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7df1649-14bc-42d6-ae9b-8f5a389e15f7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:12.535490 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7df1649-14bc-42d6-ae9b-8f5a389e15f7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:12.536249 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7df1649-14bc-42d6-ae9b-8f5a389e15f7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:12.536781 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7df1649-14bc-42d6-ae9b-8f5a389e15f7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:12.537250 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7df1649-14bc-42d6-ae9b-8f5a389e15f7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:12.769243 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f7df1649-14bc-42d6-ae9b-8f5a389e15f7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:12.778245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f7df1649-14bc-42d6-ae9b-8f5a389e15f7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:12.779729 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f7df1649-14bc-42d6-ae9b-8f5a389e15f7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:12.780939 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f7df1649-14bc-42d6-ae9b-8f5a389e15f7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.258236 Oct 24 08:55:12.781505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 228/466] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:12 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:13.219501 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-984dd893-5ca1-4ab2-9651-ab12a5492db6 req-0ef2b52a-0b0f-4b27-8e5e-e669a4595e7e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8f61c54c-9a6d-445b-8a49-056657208b85", "tag": "e9bb3baa-6f12-4847-a485-980d16c4f05b"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:55:13.230337 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-984dd893-5ca1-4ab2-9651-ab12a5492db6 req-0ef2b52a-0b0f-4b27-8e5e-e669a4595e7e service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:13.230806 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-984dd893-5ca1-4ab2-9651-ab12a5492db6 req-0ef2b52a-0b0f-4b27-8e5e-e669a4595e7e service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:13.231231 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-984dd893-5ca1-4ab2-9651-ab12a5492db6 req-0ef2b52a-0b0f-4b27-8e5e-e669a4595e7e service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:13.245734 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-984dd893-5ca1-4ab2-9651-ab12a5492db6 req-0ef2b52a-0b0f-4b27-8e5e-e669a4595e7e service nova] Creating event network-changed:e9bb3baa-6f12-4847-a485-980d16c4f05b for instance 8f61c54c-9a6d-445b-8a49-056657208b85 on hv-cinder-maste Oct 24 08:55:13.251236 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-984dd893-5ca1-4ab2-9651-ab12a5492db6 req-0ef2b52a-0b0f-4b27-8e5e-e669a4595e7e service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036138 Oct 24 08:55:13.251838 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 239/467] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:55:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:55:13.806958 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4be20645-17c6-4ed7-8365-7463cfda48f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:13.807512 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4be20645-17c6-4ed7-8365-7463cfda48f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:13.813989 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4be20645-17c6-4ed7-8365-7463cfda48f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:13.814481 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4be20645-17c6-4ed7-8365-7463cfda48f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:13.814914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4be20645-17c6-4ed7-8365-7463cfda48f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:13.815689 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4be20645-17c6-4ed7-8365-7463cfda48f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:13.816157 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4be20645-17c6-4ed7-8365-7463cfda48f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:13.816602 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4be20645-17c6-4ed7-8365-7463cfda48f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:13.957823 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4be20645-17c6-4ed7-8365-7463cfda48f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:13.968926 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4be20645-17c6-4ed7-8365-7463cfda48f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:13.970416 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4be20645-17c6-4ed7-8365-7463cfda48f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:13.971677 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4be20645-17c6-4ed7-8365-7463cfda48f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.168797 Oct 24 08:55:13.972271 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 229/468] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:13 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:14.701742 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d4f9024d-eab7-4ab8-bbb6-d60d98ff686d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:55:14.708358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d4f9024d-eab7-4ab8-bbb6-d60d98ff686d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:14.708842 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d4f9024d-eab7-4ab8-bbb6-d60d98ff686d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:14.709293 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d4f9024d-eab7-4ab8-bbb6-d60d98ff686d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:14.768205 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d4f9024d-eab7-4ab8-bbb6-d60d98ff686d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Going to try to stop instance {{(pid=130395) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 24 08:55:14.810626 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d4f9024d-eab7-4ab8-bbb6-d60d98ff686d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "POST /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/action" status: 202 len: 0 microversion: 2.1 time: 0.112859 Oct 24 08:55:14.811227 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 240/469] 10.222.0.39 () {60 vars in 1278 bytes} [Mon Oct 24 08:55:14 2022] POST /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/action => generated 0 bytes in 114 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 24 08:55:14.819391 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-033bc511-8214-4e2d-9719-657c96b7f2ee tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:14.820803 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-033bc511-8214-4e2d-9719-657c96b7f2ee tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:14.827814 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-033bc511-8214-4e2d-9719-657c96b7f2ee tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:14.828347 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-033bc511-8214-4e2d-9719-657c96b7f2ee tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:14.828789 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-033bc511-8214-4e2d-9719-657c96b7f2ee tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:14.829492 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-033bc511-8214-4e2d-9719-657c96b7f2ee tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:14.829951 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-033bc511-8214-4e2d-9719-657c96b7f2ee tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:14.830383 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-033bc511-8214-4e2d-9719-657c96b7f2ee tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:14.985488 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-033bc511-8214-4e2d-9719-657c96b7f2ee tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:14.998263 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-033bc511-8214-4e2d-9719-657c96b7f2ee tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1724 microversion: 2.1 time: 0.182630 Oct 24 08:55:15.000168 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 230/470] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:14 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1724 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:55:15.000412 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1ad3f562-b294-4538-a23a-e690e82f1b75 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:15.001430 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1ad3f562-b294-4538-a23a-e690e82f1b75 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:15.009617 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ad3f562-b294-4538-a23a-e690e82f1b75 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:15.010495 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ad3f562-b294-4538-a23a-e690e82f1b75 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:15.011179 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ad3f562-b294-4538-a23a-e690e82f1b75 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:15.012106 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ad3f562-b294-4538-a23a-e690e82f1b75 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:15.012794 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ad3f562-b294-4538-a23a-e690e82f1b75 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:15.013427 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ad3f562-b294-4538-a23a-e690e82f1b75 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:15.146931 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1ad3f562-b294-4538-a23a-e690e82f1b75 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:15.155927 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1ad3f562-b294-4538-a23a-e690e82f1b75 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:15.157641 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1ad3f562-b294-4538-a23a-e690e82f1b75 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:15.158849 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1ad3f562-b294-4538-a23a-e690e82f1b75 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.164148 Oct 24 08:55:15.159404 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 241/471] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:14 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:55:16.022677 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9c0bf180-0754-46a7-b143-9e49cedf8289 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:16.023745 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9c0bf180-0754-46a7-b143-9e49cedf8289 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:16.029953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c0bf180-0754-46a7-b143-9e49cedf8289 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:16.030448 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c0bf180-0754-46a7-b143-9e49cedf8289 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:16.030888 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c0bf180-0754-46a7-b143-9e49cedf8289 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:16.031596 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c0bf180-0754-46a7-b143-9e49cedf8289 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:16.032051 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c0bf180-0754-46a7-b143-9e49cedf8289 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:16.032505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9c0bf180-0754-46a7-b143-9e49cedf8289 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:16.170963 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9c0bf180-0754-46a7-b143-9e49cedf8289 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:16.180114 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9c0bf180-0754-46a7-b143-9e49cedf8289 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1724 microversion: 2.1 time: 0.160554 Oct 24 08:55:16.180848 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 231/472] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:16 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1724 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:16.186415 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-958203ee-9255-4b96-a26a-0795caf7e8b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:16.187053 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-958203ee-9255-4b96-a26a-0795caf7e8b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:16.194408 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-958203ee-9255-4b96-a26a-0795caf7e8b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:16.194956 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-958203ee-9255-4b96-a26a-0795caf7e8b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:16.195404 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-958203ee-9255-4b96-a26a-0795caf7e8b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:16.196109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-958203ee-9255-4b96-a26a-0795caf7e8b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:16.196598 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-958203ee-9255-4b96-a26a-0795caf7e8b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:16.197026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-958203ee-9255-4b96-a26a-0795caf7e8b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:16.332900 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-958203ee-9255-4b96-a26a-0795caf7e8b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:16.343217 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-958203ee-9255-4b96-a26a-0795caf7e8b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:16.344671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-958203ee-9255-4b96-a26a-0795caf7e8b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:16.345833 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-958203ee-9255-4b96-a26a-0795caf7e8b6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.164074 Oct 24 08:55:16.346395 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 242/473] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:16 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:55:17.204148 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7fd985f7-efdc-4ab6-b772-13c1322e96d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:17.204769 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7fd985f7-efdc-4ab6-b772-13c1322e96d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:17.211838 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fd985f7-efdc-4ab6-b772-13c1322e96d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:17.212356 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fd985f7-efdc-4ab6-b772-13c1322e96d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:17.212798 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fd985f7-efdc-4ab6-b772-13c1322e96d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:17.213531 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fd985f7-efdc-4ab6-b772-13c1322e96d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:17.213981 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fd985f7-efdc-4ab6-b772-13c1322e96d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:17.214408 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fd985f7-efdc-4ab6-b772-13c1322e96d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:17.345576 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7fd985f7-efdc-4ab6-b772-13c1322e96d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:17.354096 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7fd985f7-efdc-4ab6-b772-13c1322e96d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1724 microversion: 2.1 time: 0.153843 Oct 24 08:55:17.354668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 232/474] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:17 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1724 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:17.373086 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-34553e20-f71e-4506-b474-38bc221482ee tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:17.373697 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-34553e20-f71e-4506-b474-38bc221482ee tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:17.380126 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-34553e20-f71e-4506-b474-38bc221482ee tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:17.380660 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-34553e20-f71e-4506-b474-38bc221482ee tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:17.381112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-34553e20-f71e-4506-b474-38bc221482ee tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:17.381806 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-34553e20-f71e-4506-b474-38bc221482ee tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:17.382952 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-34553e20-f71e-4506-b474-38bc221482ee tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:17.383388 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-34553e20-f71e-4506-b474-38bc221482ee tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:17.778843 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-34553e20-f71e-4506-b474-38bc221482ee tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:17.790179 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-34553e20-f71e-4506-b474-38bc221482ee tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:17.791634 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-34553e20-f71e-4506-b474-38bc221482ee tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:17.792881 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-34553e20-f71e-4506-b474-38bc221482ee tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.423680 Oct 24 08:55:17.793495 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 243/475] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:17 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:18.378240 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2d238fcb-db53-4615-9ce6-c3711e3cbf9d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:18.378830 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2d238fcb-db53-4615-9ce6-c3711e3cbf9d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:18.385999 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2d238fcb-db53-4615-9ce6-c3711e3cbf9d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:18.386476 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2d238fcb-db53-4615-9ce6-c3711e3cbf9d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:18.387050 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2d238fcb-db53-4615-9ce6-c3711e3cbf9d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:18.387847 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2d238fcb-db53-4615-9ce6-c3711e3cbf9d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:18.388031 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2d238fcb-db53-4615-9ce6-c3711e3cbf9d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:18.388548 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2d238fcb-db53-4615-9ce6-c3711e3cbf9d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:18.521570 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2d238fcb-db53-4615-9ce6-c3711e3cbf9d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:18.531191 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2d238fcb-db53-4615-9ce6-c3711e3cbf9d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1724 microversion: 2.1 time: 0.156721 Oct 24 08:55:18.531938 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 233/476] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:18 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1724 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:55:18.825804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f5781c83-3281-4f94-8b9f-6f7f859ade65 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:18.826359 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f5781c83-3281-4f94-8b9f-6f7f859ade65 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:18.832717 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f5781c83-3281-4f94-8b9f-6f7f859ade65 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:18.833197 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f5781c83-3281-4f94-8b9f-6f7f859ade65 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:18.833634 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f5781c83-3281-4f94-8b9f-6f7f859ade65 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:18.834328 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f5781c83-3281-4f94-8b9f-6f7f859ade65 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:18.834777 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f5781c83-3281-4f94-8b9f-6f7f859ade65 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:18.835234 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f5781c83-3281-4f94-8b9f-6f7f859ade65 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:18.977558 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f5781c83-3281-4f94-8b9f-6f7f859ade65 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:18.986469 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f5781c83-3281-4f94-8b9f-6f7f859ade65 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:18.988000 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f5781c83-3281-4f94-8b9f-6f7f859ade65 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:18.989302 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f5781c83-3281-4f94-8b9f-6f7f859ade65 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.167478 Oct 24 08:55:18.990123 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 244/477] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:18 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:55:19.556824 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-dd8d389e-f51b-499a-9cd7-65c02bb05f70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:19.557537 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-dd8d389e-f51b-499a-9cd7-65c02bb05f70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:19.564489 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dd8d389e-f51b-499a-9cd7-65c02bb05f70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:19.564981 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dd8d389e-f51b-499a-9cd7-65c02bb05f70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:19.565422 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dd8d389e-f51b-499a-9cd7-65c02bb05f70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:19.566135 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dd8d389e-f51b-499a-9cd7-65c02bb05f70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:19.566598 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dd8d389e-f51b-499a-9cd7-65c02bb05f70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:19.567025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dd8d389e-f51b-499a-9cd7-65c02bb05f70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:19.703457 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-dd8d389e-f51b-499a-9cd7-65c02bb05f70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:19.716322 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-dd8d389e-f51b-499a-9cd7-65c02bb05f70 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1724 microversion: 2.1 time: 0.163988 Oct 24 08:55:19.716931 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 234/478] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:19 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1724 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:20.016154 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-2d17cd98-fbbc-47b6-9810-54cce077153e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:20.016898 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-2d17cd98-fbbc-47b6-9810-54cce077153e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:20.023553 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2d17cd98-fbbc-47b6-9810-54cce077153e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:20.024037 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2d17cd98-fbbc-47b6-9810-54cce077153e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:20.024525 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2d17cd98-fbbc-47b6-9810-54cce077153e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:20.025219 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2d17cd98-fbbc-47b6-9810-54cce077153e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:20.025674 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2d17cd98-fbbc-47b6-9810-54cce077153e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:20.026102 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2d17cd98-fbbc-47b6-9810-54cce077153e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:20.165045 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-2d17cd98-fbbc-47b6-9810-54cce077153e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:20.174169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-2d17cd98-fbbc-47b6-9810-54cce077153e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:20.175596 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-2d17cd98-fbbc-47b6-9810-54cce077153e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:20.176813 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-2d17cd98-fbbc-47b6-9810-54cce077153e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.164547 Oct 24 08:55:20.177366 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 245/479] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:20 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:20.740847 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e97d9c34-ef82-4ddd-a149-81f814602b1e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:20.741697 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e97d9c34-ef82-4ddd-a149-81f814602b1e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:20.748019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e97d9c34-ef82-4ddd-a149-81f814602b1e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:20.748474 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e97d9c34-ef82-4ddd-a149-81f814602b1e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:20.748906 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e97d9c34-ef82-4ddd-a149-81f814602b1e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:20.749607 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e97d9c34-ef82-4ddd-a149-81f814602b1e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:20.750055 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e97d9c34-ef82-4ddd-a149-81f814602b1e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:20.750475 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e97d9c34-ef82-4ddd-a149-81f814602b1e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:20.891288 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e97d9c34-ef82-4ddd-a149-81f814602b1e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:20.900623 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e97d9c34-ef82-4ddd-a149-81f814602b1e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1724 microversion: 2.1 time: 0.163845 Oct 24 08:55:20.901244 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 235/480] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:20 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1724 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:21.206725 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1918770b-1b2e-4a00-9f51-905b3c6cfcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:21.207738 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1918770b-1b2e-4a00-9f51-905b3c6cfcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:21.219408 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1918770b-1b2e-4a00-9f51-905b3c6cfcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:21.220254 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1918770b-1b2e-4a00-9f51-905b3c6cfcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:21.220993 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1918770b-1b2e-4a00-9f51-905b3c6cfcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:21.222228 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1918770b-1b2e-4a00-9f51-905b3c6cfcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:21.223006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1918770b-1b2e-4a00-9f51-905b3c6cfcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:21.223694 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1918770b-1b2e-4a00-9f51-905b3c6cfcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:21.388230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1918770b-1b2e-4a00-9f51-905b3c6cfcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:21.397671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1918770b-1b2e-4a00-9f51-905b3c6cfcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:21.399728 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1918770b-1b2e-4a00-9f51-905b3c6cfcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:21.400993 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1918770b-1b2e-4a00-9f51-905b3c6cfcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.201357 Oct 24 08:55:21.401562 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 246/481] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:21 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:21.924501 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9a65d678-9f72-416e-aabd-6e0bbd058eb9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:21.925236 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9a65d678-9f72-416e-aabd-6e0bbd058eb9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:21.933248 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9a65d678-9f72-416e-aabd-6e0bbd058eb9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:21.933734 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9a65d678-9f72-416e-aabd-6e0bbd058eb9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:21.934170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9a65d678-9f72-416e-aabd-6e0bbd058eb9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:21.934898 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9a65d678-9f72-416e-aabd-6e0bbd058eb9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:21.935349 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9a65d678-9f72-416e-aabd-6e0bbd058eb9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:21.935767 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9a65d678-9f72-416e-aabd-6e0bbd058eb9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:22.122531 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9a65d678-9f72-416e-aabd-6e0bbd058eb9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:22.130809 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9a65d678-9f72-416e-aabd-6e0bbd058eb9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1724 microversion: 2.1 time: 0.210298 Oct 24 08:55:22.131439 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 236/482] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:21 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1724 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:22.426004 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-183cc3cd-b0ac-4d53-8df2-e47cbf6ac1e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:22.426652 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-183cc3cd-b0ac-4d53-8df2-e47cbf6ac1e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:22.433632 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-183cc3cd-b0ac-4d53-8df2-e47cbf6ac1e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:22.434197 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-183cc3cd-b0ac-4d53-8df2-e47cbf6ac1e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:22.434641 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-183cc3cd-b0ac-4d53-8df2-e47cbf6ac1e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:22.435361 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-183cc3cd-b0ac-4d53-8df2-e47cbf6ac1e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:22.435821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-183cc3cd-b0ac-4d53-8df2-e47cbf6ac1e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:22.436281 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-183cc3cd-b0ac-4d53-8df2-e47cbf6ac1e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:22.575643 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-183cc3cd-b0ac-4d53-8df2-e47cbf6ac1e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:22.584587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-183cc3cd-b0ac-4d53-8df2-e47cbf6ac1e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:22.586656 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-183cc3cd-b0ac-4d53-8df2-e47cbf6ac1e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:22.587755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-183cc3cd-b0ac-4d53-8df2-e47cbf6ac1e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.165704 Oct 24 08:55:22.588352 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 247/483] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:22 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:22.800068 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-b13a1b98-9775-47c6-9be2-c9bfdbd69a71 req-1539357b-b83a-4801-a2b5-63ae0978ffcf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "71e106ba-ad54-4e9c-8e42-9477c8db8cf6", "name": "network-vif-plugged", "status": "completed", "tag": "b8754c97-b769-418c-b461-714a5809d621"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:55:22.810554 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-b13a1b98-9775-47c6-9be2-c9bfdbd69a71 req-1539357b-b83a-4801-a2b5-63ae0978ffcf service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:22.811035 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-b13a1b98-9775-47c6-9be2-c9bfdbd69a71 req-1539357b-b83a-4801-a2b5-63ae0978ffcf service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:22.811462 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-b13a1b98-9775-47c6-9be2-c9bfdbd69a71 req-1539357b-b83a-4801-a2b5-63ae0978ffcf service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:22.823709 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-b13a1b98-9775-47c6-9be2-c9bfdbd69a71 req-1539357b-b83a-4801-a2b5-63ae0978ffcf service nova] Creating event network-vif-plugged:b8754c97-b769-418c-b461-714a5809d621 for instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 on hv-cinder-maste Oct 24 08:55:22.829014 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-b13a1b98-9775-47c6-9be2-c9bfdbd69a71 req-1539357b-b83a-4801-a2b5-63ae0978ffcf service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032645 Oct 24 08:55:22.829993 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 237/484] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:55:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:55:23.154783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f2398c00-aa2f-4e6a-b223-e20e68627cd9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:23.155391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f2398c00-aa2f-4e6a-b223-e20e68627cd9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:23.161332 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f2398c00-aa2f-4e6a-b223-e20e68627cd9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:23.161816 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f2398c00-aa2f-4e6a-b223-e20e68627cd9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:23.162300 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f2398c00-aa2f-4e6a-b223-e20e68627cd9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:23.163061 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f2398c00-aa2f-4e6a-b223-e20e68627cd9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:23.164168 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f2398c00-aa2f-4e6a-b223-e20e68627cd9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:23.164725 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f2398c00-aa2f-4e6a-b223-e20e68627cd9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:23.307834 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f2398c00-aa2f-4e6a-b223-e20e68627cd9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:23.316679 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f2398c00-aa2f-4e6a-b223-e20e68627cd9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1724 microversion: 2.1 time: 0.165660 Oct 24 08:55:23.317256 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 248/485] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:23 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1724 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:23.616668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-d7709208-2770-48af-9e45-d9cdd6101607 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:23.618115 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-d7709208-2770-48af-9e45-d9cdd6101607 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:23.624614 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7709208-2770-48af-9e45-d9cdd6101607 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:23.625096 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7709208-2770-48af-9e45-d9cdd6101607 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:23.625518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7709208-2770-48af-9e45-d9cdd6101607 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:23.626209 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7709208-2770-48af-9e45-d9cdd6101607 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:23.626649 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7709208-2770-48af-9e45-d9cdd6101607 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:23.627063 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7709208-2770-48af-9e45-d9cdd6101607 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:23.762799 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d7709208-2770-48af-9e45-d9cdd6101607 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:23.771237 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d7709208-2770-48af-9e45-d9cdd6101607 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:23.773438 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d7709208-2770-48af-9e45-d9cdd6101607 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:23.775008 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-d7709208-2770-48af-9e45-d9cdd6101607 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.163790 Oct 24 08:55:23.775556 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 238/486] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:23 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:24.339998 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-45736fd8-0477-4d6a-aa2c-ee12b501a4d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:24.340705 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-45736fd8-0477-4d6a-aa2c-ee12b501a4d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:24.347307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-45736fd8-0477-4d6a-aa2c-ee12b501a4d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:24.347792 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-45736fd8-0477-4d6a-aa2c-ee12b501a4d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:24.348255 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-45736fd8-0477-4d6a-aa2c-ee12b501a4d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:24.348959 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-45736fd8-0477-4d6a-aa2c-ee12b501a4d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:24.349414 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-45736fd8-0477-4d6a-aa2c-ee12b501a4d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:24.349883 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-45736fd8-0477-4d6a-aa2c-ee12b501a4d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:24.489833 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-45736fd8-0477-4d6a-aa2c-ee12b501a4d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:24.499014 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-45736fd8-0477-4d6a-aa2c-ee12b501a4d4 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1724 microversion: 2.1 time: 0.162917 Oct 24 08:55:24.499533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 249/487] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:24 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1724 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:24.576366 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0c3396f0-95b2-42df-9963-99279a67a761 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c52e4f61-dd63-48b5-a68c-0294cbee81a6"}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:55:24.578771 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-0c3396f0-95b2-42df-9963-99279a67a761 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:24.584830 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0c3396f0-95b2-42df-9963-99279a67a761 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:24.585818 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0c3396f0-95b2-42df-9963-99279a67a761 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:24.586792 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0c3396f0-95b2-42df-9963-99279a67a761 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:24.586934 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0c3396f0-95b2-42df-9963-99279a67a761 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:24.587484 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0c3396f0-95b2-42df-9963-99279a67a761 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:24.587819 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0c3396f0-95b2-42df-9963-99279a67a761 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:24.805744 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-78a6b92e-dc61-4aea-b3b9-b823d5110f14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:24.806358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-78a6b92e-dc61-4aea-b3b9-b823d5110f14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:24.810614 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:24.812771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-78a6b92e-dc61-4aea-b3b9-b823d5110f14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:24.813240 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-78a6b92e-dc61-4aea-b3b9-b823d5110f14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:24.813826 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-78a6b92e-dc61-4aea-b3b9-b823d5110f14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:24.814582 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-78a6b92e-dc61-4aea-b3b9-b823d5110f14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:24.815002 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-78a6b92e-dc61-4aea-b3b9-b823d5110f14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:24.815431 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-78a6b92e-dc61-4aea-b3b9-b823d5110f14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:24.947698 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-0c3396f0-95b2-42df-9963-99279a67a761 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "POST /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.375202 Oct 24 08:55:24.948514 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 239/488] 10.222.0.39 () {60 vars in 1323 bytes} [Mon Oct 24 08:55:24 2022] POST /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/os-volume_attachments => generated 194 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:55:24.964963 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-78a6b92e-dc61-4aea-b3b9-b823d5110f14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:24.975142 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-78a6b92e-dc61-4aea-b3b9-b823d5110f14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:24.977470 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-78a6b92e-dc61-4aea-b3b9-b823d5110f14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:24.978859 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-78a6b92e-dc61-4aea-b3b9-b823d5110f14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.177580 Oct 24 08:55:24.980026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 250/489] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:24 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:55:25.523407 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-fc9e90b2-99ba-42aa-b7b5-23680a327fc1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:25.524113 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-fc9e90b2-99ba-42aa-b7b5-23680a327fc1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:25.530916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fc9e90b2-99ba-42aa-b7b5-23680a327fc1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:25.531511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fc9e90b2-99ba-42aa-b7b5-23680a327fc1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:25.531986 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fc9e90b2-99ba-42aa-b7b5-23680a327fc1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:25.532771 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fc9e90b2-99ba-42aa-b7b5-23680a327fc1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:25.533304 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fc9e90b2-99ba-42aa-b7b5-23680a327fc1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:25.534017 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fc9e90b2-99ba-42aa-b7b5-23680a327fc1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:25.673626 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fc9e90b2-99ba-42aa-b7b5-23680a327fc1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:25.683528 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-fc9e90b2-99ba-42aa-b7b5-23680a327fc1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1724 microversion: 2.1 time: 0.164834 Oct 24 08:55:25.684092 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 240/490] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:25 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1724 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:26.008521 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-41671b89-8e8b-4ca0-a3b6-60dc14b42be8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:26.009295 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-41671b89-8e8b-4ca0-a3b6-60dc14b42be8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:26.015608 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-41671b89-8e8b-4ca0-a3b6-60dc14b42be8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:26.016102 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-41671b89-8e8b-4ca0-a3b6-60dc14b42be8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:26.016608 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-41671b89-8e8b-4ca0-a3b6-60dc14b42be8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:26.017323 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-41671b89-8e8b-4ca0-a3b6-60dc14b42be8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:26.017777 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-41671b89-8e8b-4ca0-a3b6-60dc14b42be8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:26.018205 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-41671b89-8e8b-4ca0-a3b6-60dc14b42be8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:26.172077 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-41671b89-8e8b-4ca0-a3b6-60dc14b42be8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:26.180811 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-41671b89-8e8b-4ca0-a3b6-60dc14b42be8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:26.182255 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-41671b89-8e8b-4ca0-a3b6-60dc14b42be8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:26.183406 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-41671b89-8e8b-4ca0-a3b6-60dc14b42be8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1808 microversion: 2.20 time: 0.180434 Oct 24 08:55:26.183968 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 251/491] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:26 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1808 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:26.706793 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-38ff9650-3957-4eaf-b8b3-f18397d5f974 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:26.707428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-38ff9650-3957-4eaf-b8b3-f18397d5f974 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:26.714619 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-38ff9650-3957-4eaf-b8b3-f18397d5f974 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:26.715106 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-38ff9650-3957-4eaf-b8b3-f18397d5f974 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:26.715542 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-38ff9650-3957-4eaf-b8b3-f18397d5f974 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:26.716266 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-38ff9650-3957-4eaf-b8b3-f18397d5f974 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:26.716728 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-38ff9650-3957-4eaf-b8b3-f18397d5f974 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:26.717153 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-38ff9650-3957-4eaf-b8b3-f18397d5f974 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:26.848615 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-38ff9650-3957-4eaf-b8b3-f18397d5f974 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:26.857104 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-38ff9650-3957-4eaf-b8b3-f18397d5f974 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1724 microversion: 2.1 time: 0.154175 Oct 24 08:55:26.857827 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 241/492] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:26 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1724 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:27.209154 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-70b4b0fa-2075-4ca4-ab7a-4b59ec0c357e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:27.209782 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-70b4b0fa-2075-4ca4-ab7a-4b59ec0c357e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:27.217344 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-70b4b0fa-2075-4ca4-ab7a-4b59ec0c357e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:27.217817 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-70b4b0fa-2075-4ca4-ab7a-4b59ec0c357e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:27.218246 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-70b4b0fa-2075-4ca4-ab7a-4b59ec0c357e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:27.218932 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-70b4b0fa-2075-4ca4-ab7a-4b59ec0c357e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:27.219378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-70b4b0fa-2075-4ca4-ab7a-4b59ec0c357e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:27.219794 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-70b4b0fa-2075-4ca4-ab7a-4b59ec0c357e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:27.355021 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-70b4b0fa-2075-4ca4-ab7a-4b59ec0c357e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:27.363599 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-70b4b0fa-2075-4ca4-ab7a-4b59ec0c357e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:27.365332 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-70b4b0fa-2075-4ca4-ab7a-4b59ec0c357e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:27.366926 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-70b4b0fa-2075-4ca4-ab7a-4b59ec0c357e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1795 microversion: 2.20 time: 0.161252 Oct 24 08:55:27.367487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 252/493] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:55:27 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1795 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:55:27.882441 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ffe74f2b-3903-4bc8-83a7-565301437ed7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:27.883095 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ffe74f2b-3903-4bc8-83a7-565301437ed7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:27.889643 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ffe74f2b-3903-4bc8-83a7-565301437ed7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:27.890125 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ffe74f2b-3903-4bc8-83a7-565301437ed7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:27.890568 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ffe74f2b-3903-4bc8-83a7-565301437ed7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:27.891352 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ffe74f2b-3903-4bc8-83a7-565301437ed7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:27.891824 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ffe74f2b-3903-4bc8-83a7-565301437ed7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:27.892283 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ffe74f2b-3903-4bc8-83a7-565301437ed7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:28.025922 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ffe74f2b-3903-4bc8-83a7-565301437ed7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:28.035865 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ffe74f2b-3903-4bc8-83a7-565301437ed7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1724 microversion: 2.1 time: 0.157851 Oct 24 08:55:28.036466 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 242/494] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:27 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1724 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:29.059287 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c49527b2-26f1-48a7-a2eb-9b7cb140e97f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:29.059882 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c49527b2-26f1-48a7-a2eb-9b7cb140e97f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:29.066318 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c49527b2-26f1-48a7-a2eb-9b7cb140e97f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:29.066792 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c49527b2-26f1-48a7-a2eb-9b7cb140e97f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:29.067225 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c49527b2-26f1-48a7-a2eb-9b7cb140e97f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:29.067915 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c49527b2-26f1-48a7-a2eb-9b7cb140e97f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:29.068400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c49527b2-26f1-48a7-a2eb-9b7cb140e97f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:29.068879 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c49527b2-26f1-48a7-a2eb-9b7cb140e97f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:29.074727 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:29.205944 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c49527b2-26f1-48a7-a2eb-9b7cb140e97f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:29.214373 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c49527b2-26f1-48a7-a2eb-9b7cb140e97f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1724 microversion: 2.1 time: 0.158868 Oct 24 08:55:29.214945 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 253/495] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:29 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1724 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:30.238857 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-bd67f37f-d6a2-4009-8561-98e8e133ba82 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:30.239761 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-bd67f37f-d6a2-4009-8561-98e8e133ba82 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:30.246655 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bd67f37f-d6a2-4009-8561-98e8e133ba82 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:30.247087 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bd67f37f-d6a2-4009-8561-98e8e133ba82 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:30.247518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bd67f37f-d6a2-4009-8561-98e8e133ba82 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:30.248252 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bd67f37f-d6a2-4009-8561-98e8e133ba82 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:30.248701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bd67f37f-d6a2-4009-8561-98e8e133ba82 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:30.249121 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bd67f37f-d6a2-4009-8561-98e8e133ba82 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:30.391955 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-bd67f37f-d6a2-4009-8561-98e8e133ba82 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:30.400850 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-bd67f37f-d6a2-4009-8561-98e8e133ba82 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1724 microversion: 2.1 time: 0.165861 Oct 24 08:55:30.401417 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 243/496] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:30 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1724 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:31.424923 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-deacc42a-527d-4514-86e9-9eef8998da26 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:31.425524 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-deacc42a-527d-4514-86e9-9eef8998da26 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:31.432055 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-deacc42a-527d-4514-86e9-9eef8998da26 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:31.432561 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-deacc42a-527d-4514-86e9-9eef8998da26 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:31.432992 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-deacc42a-527d-4514-86e9-9eef8998da26 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:31.433679 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-deacc42a-527d-4514-86e9-9eef8998da26 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:31.434122 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-deacc42a-527d-4514-86e9-9eef8998da26 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:31.434539 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-deacc42a-527d-4514-86e9-9eef8998da26 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:31.564343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-deacc42a-527d-4514-86e9-9eef8998da26 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:31.573824 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-deacc42a-527d-4514-86e9-9eef8998da26 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1701 microversion: 2.1 time: 0.152704 Oct 24 08:55:31.574389 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 254/497] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:31 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1701 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:31.596944 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-69527e40-27a8-4e90-95c3-6d7b4604657f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:55:31.605291 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-69527e40-27a8-4e90-95c3-6d7b4604657f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:31.605968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-69527e40-27a8-4e90-95c3-6d7b4604657f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:31.606413 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-69527e40-27a8-4e90-95c3-6d7b4604657f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:31.669213 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-69527e40-27a8-4e90-95c3-6d7b4604657f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Going to try to start instance {{(pid=130396) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 24 08:55:31.710556 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-69527e40-27a8-4e90-95c3-6d7b4604657f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "POST /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/action" status: 202 len: 0 microversion: 2.1 time: 0.116952 Oct 24 08:55:31.711179 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 244/498] 10.222.0.39 () {60 vars in 1278 bytes} [Mon Oct 24 08:55:31 2022] POST /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/action => generated 0 bytes in 119 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 08:55:31.719576 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1fd7fcb3-cf8b-43fc-aa77-62192e75c13e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:31.720215 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1fd7fcb3-cf8b-43fc-aa77-62192e75c13e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:31.727131 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1fd7fcb3-cf8b-43fc-aa77-62192e75c13e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:31.727616 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1fd7fcb3-cf8b-43fc-aa77-62192e75c13e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:31.728052 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1fd7fcb3-cf8b-43fc-aa77-62192e75c13e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:31.728821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1fd7fcb3-cf8b-43fc-aa77-62192e75c13e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:31.729279 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1fd7fcb3-cf8b-43fc-aa77-62192e75c13e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:31.729698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1fd7fcb3-cf8b-43fc-aa77-62192e75c13e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:31.865863 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1fd7fcb3-cf8b-43fc-aa77-62192e75c13e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:31.874058 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1fd7fcb3-cf8b-43fc-aa77-62192e75c13e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1710 microversion: 2.1 time: 0.158441 Oct 24 08:55:31.874623 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 255/499] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:31 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1710 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:32.900355 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b0d1f2a0-6600-49cf-a2b0-a4ded5b1bc4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:32.901066 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b0d1f2a0-6600-49cf-a2b0-a4ded5b1bc4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:32.908827 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0d1f2a0-6600-49cf-a2b0-a4ded5b1bc4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:32.909325 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0d1f2a0-6600-49cf-a2b0-a4ded5b1bc4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:32.909759 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0d1f2a0-6600-49cf-a2b0-a4ded5b1bc4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:32.910648 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0d1f2a0-6600-49cf-a2b0-a4ded5b1bc4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:32.911259 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0d1f2a0-6600-49cf-a2b0-a4ded5b1bc4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:32.911694 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0d1f2a0-6600-49cf-a2b0-a4ded5b1bc4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:33.054460 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b0d1f2a0-6600-49cf-a2b0-a4ded5b1bc4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:33.063114 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b0d1f2a0-6600-49cf-a2b0-a4ded5b1bc4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1710 microversion: 2.1 time: 0.168474 Oct 24 08:55:33.063697 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 245/500] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:32 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1710 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:33.591441 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-0e3c9eb5-afdb-4c35-b1e9-59b9af1b084b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:55:33.593855 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-0e3c9eb5-afdb-4c35-b1e9-59b9af1b084b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:33.601513 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0e3c9eb5-afdb-4c35-b1e9-59b9af1b084b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:33.602011 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0e3c9eb5-afdb-4c35-b1e9-59b9af1b084b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:33.602449 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0e3c9eb5-afdb-4c35-b1e9-59b9af1b084b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:33.603164 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0e3c9eb5-afdb-4c35-b1e9-59b9af1b084b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:33.603620 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0e3c9eb5-afdb-4c35-b1e9-59b9af1b084b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:33.604042 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0e3c9eb5-afdb-4c35-b1e9-59b9af1b084b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:33.628009 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-0e3c9eb5-afdb-4c35-b1e9-59b9af1b084b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lazy-loading 'flavor' on Instance uuid 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:55:33.754560 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-0e3c9eb5-afdb-4c35-b1e9-59b9af1b084b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "POST /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/action" status: 200 len: 32 microversion: 2.1 time: 0.167218 Oct 24 08:55:33.755166 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 256/501] 10.222.0.39 () {60 vars in 1278 bytes} [Mon Oct 24 08:55:33 2022] POST /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/action => generated 32 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 24 08:55:33.765006 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2986dd69-0700-4796-bcef-55e1491ae8f4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:33.765321 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2986dd69-0700-4796-bcef-55e1491ae8f4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:33.771426 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2986dd69-0700-4796-bcef-55e1491ae8f4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:33.772878 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2986dd69-0700-4796-bcef-55e1491ae8f4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:33.773226 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2986dd69-0700-4796-bcef-55e1491ae8f4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:33.774053 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2986dd69-0700-4796-bcef-55e1491ae8f4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:33.774512 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2986dd69-0700-4796-bcef-55e1491ae8f4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:33.774948 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2986dd69-0700-4796-bcef-55e1491ae8f4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:33.911850 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2986dd69-0700-4796-bcef-55e1491ae8f4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:33.920779 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2986dd69-0700-4796-bcef-55e1491ae8f4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.159819 Oct 24 08:55:33.921358 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 246/502] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:33 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:34.087959 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f87a9fcc-882b-4588-8e69-d2be02688ad9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:34.088700 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f87a9fcc-882b-4588-8e69-d2be02688ad9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:34.095178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f87a9fcc-882b-4588-8e69-d2be02688ad9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:34.095675 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f87a9fcc-882b-4588-8e69-d2be02688ad9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:34.096123 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f87a9fcc-882b-4588-8e69-d2be02688ad9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:34.096942 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f87a9fcc-882b-4588-8e69-d2be02688ad9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:34.097400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f87a9fcc-882b-4588-8e69-d2be02688ad9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:34.097831 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f87a9fcc-882b-4588-8e69-d2be02688ad9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:34.246118 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f87a9fcc-882b-4588-8e69-d2be02688ad9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:34.257054 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f87a9fcc-882b-4588-8e69-d2be02688ad9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1710 microversion: 2.1 time: 0.173141 Oct 24 08:55:34.258032 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 257/503] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:34 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1710 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:34.944702 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-55ec63db-b47b-4291-a6fe-3b1e9d42e7ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:34.945338 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-55ec63db-b47b-4291-a6fe-3b1e9d42e7ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:34.951795 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-55ec63db-b47b-4291-a6fe-3b1e9d42e7ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:34.952313 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-55ec63db-b47b-4291-a6fe-3b1e9d42e7ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:34.952961 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-55ec63db-b47b-4291-a6fe-3b1e9d42e7ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:34.953658 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-55ec63db-b47b-4291-a6fe-3b1e9d42e7ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:34.954109 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-55ec63db-b47b-4291-a6fe-3b1e9d42e7ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:34.955074 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-55ec63db-b47b-4291-a6fe-3b1e9d42e7ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:35.090481 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-55ec63db-b47b-4291-a6fe-3b1e9d42e7ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:35.098979 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-55ec63db-b47b-4291-a6fe-3b1e9d42e7ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.158098 Oct 24 08:55:35.099694 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 247/504] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:34 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:35.281535 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a5d54458-2d75-4852-940c-c7c524df0b14 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:35.282196 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a5d54458-2d75-4852-940c-c7c524df0b14 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:35.289292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5d54458-2d75-4852-940c-c7c524df0b14 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:35.289772 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5d54458-2d75-4852-940c-c7c524df0b14 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:35.290213 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5d54458-2d75-4852-940c-c7c524df0b14 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:35.290911 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5d54458-2d75-4852-940c-c7c524df0b14 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:35.291368 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5d54458-2d75-4852-940c-c7c524df0b14 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:35.291791 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5d54458-2d75-4852-940c-c7c524df0b14 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:35.427586 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a5d54458-2d75-4852-940c-c7c524df0b14 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:35.436240 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a5d54458-2d75-4852-940c-c7c524df0b14 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1714 microversion: 2.1 time: 0.158251 Oct 24 08:55:35.436519 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 258/505] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:35 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1714 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:36.124109 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-68aa7528-7096-4399-b3c9-99532a8af4e6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:36.124763 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-68aa7528-7096-4399-b3c9-99532a8af4e6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:36.131454 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-68aa7528-7096-4399-b3c9-99532a8af4e6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:36.132574 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-68aa7528-7096-4399-b3c9-99532a8af4e6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:36.133018 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-68aa7528-7096-4399-b3c9-99532a8af4e6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:36.133801 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-68aa7528-7096-4399-b3c9-99532a8af4e6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:36.134266 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-68aa7528-7096-4399-b3c9-99532a8af4e6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:36.134693 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-68aa7528-7096-4399-b3c9-99532a8af4e6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:36.269119 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-68aa7528-7096-4399-b3c9-99532a8af4e6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:36.277579 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-68aa7528-7096-4399-b3c9-99532a8af4e6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.157542 Oct 24 08:55:36.278146 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 248/506] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:36 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:37.301558 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-4d2be340-fcc1-4080-9351-7c15e02b4b6d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:37.302162 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-4d2be340-fcc1-4080-9351-7c15e02b4b6d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:37.309981 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d2be340-fcc1-4080-9351-7c15e02b4b6d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:37.310502 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d2be340-fcc1-4080-9351-7c15e02b4b6d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:37.310941 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d2be340-fcc1-4080-9351-7c15e02b4b6d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:37.311641 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d2be340-fcc1-4080-9351-7c15e02b4b6d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:37.312091 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d2be340-fcc1-4080-9351-7c15e02b4b6d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:37.312547 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d2be340-fcc1-4080-9351-7c15e02b4b6d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:37.448351 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-4d2be340-fcc1-4080-9351-7c15e02b4b6d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:37.456770 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-4d2be340-fcc1-4080-9351-7c15e02b4b6d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.159035 Oct 24 08:55:37.457339 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 259/507] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:37 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:38.481185 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-17cde8fd-6a0a-4fb0-8f9a-0ff39aafbaa8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:38.481791 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-17cde8fd-6a0a-4fb0-8f9a-0ff39aafbaa8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:38.488566 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-17cde8fd-6a0a-4fb0-8f9a-0ff39aafbaa8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:38.489059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-17cde8fd-6a0a-4fb0-8f9a-0ff39aafbaa8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:38.489761 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-17cde8fd-6a0a-4fb0-8f9a-0ff39aafbaa8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:38.490479 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-17cde8fd-6a0a-4fb0-8f9a-0ff39aafbaa8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:38.491541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-17cde8fd-6a0a-4fb0-8f9a-0ff39aafbaa8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:38.491977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-17cde8fd-6a0a-4fb0-8f9a-0ff39aafbaa8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:38.632628 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-17cde8fd-6a0a-4fb0-8f9a-0ff39aafbaa8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:38.641001 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-17cde8fd-6a0a-4fb0-8f9a-0ff39aafbaa8 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.163768 Oct 24 08:55:38.641640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 249/508] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:38 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:39.665608 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ba31a1c1-de5d-48e9-8cce-d3277be31d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:39.666249 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ba31a1c1-de5d-48e9-8cce-d3277be31d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:39.672580 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ba31a1c1-de5d-48e9-8cce-d3277be31d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:39.673061 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ba31a1c1-de5d-48e9-8cce-d3277be31d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:39.673523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ba31a1c1-de5d-48e9-8cce-d3277be31d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:39.674219 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ba31a1c1-de5d-48e9-8cce-d3277be31d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:39.674670 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ba31a1c1-de5d-48e9-8cce-d3277be31d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:39.675092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ba31a1c1-de5d-48e9-8cce-d3277be31d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:39.804268 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ba31a1c1-de5d-48e9-8cce-d3277be31d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:39.812441 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ba31a1c1-de5d-48e9-8cce-d3277be31d93 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.151282 Oct 24 08:55:39.813011 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 260/509] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:39 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:40.836201 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-405def6d-3443-4a9e-838c-922e15c45a9f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:40.837472 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-405def6d-3443-4a9e-838c-922e15c45a9f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:40.843961 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-405def6d-3443-4a9e-838c-922e15c45a9f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:40.844477 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-405def6d-3443-4a9e-838c-922e15c45a9f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:40.844919 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-405def6d-3443-4a9e-838c-922e15c45a9f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:40.845616 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-405def6d-3443-4a9e-838c-922e15c45a9f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:40.846072 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-405def6d-3443-4a9e-838c-922e15c45a9f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:40.846507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-405def6d-3443-4a9e-838c-922e15c45a9f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:40.982052 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-405def6d-3443-4a9e-838c-922e15c45a9f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:40.990549 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-405def6d-3443-4a9e-838c-922e15c45a9f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.158199 Oct 24 08:55:40.991159 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 250/510] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:40 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:42.015335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e42a89d6-075f-4849-88a9-5441edb70777 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:42.015944 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e42a89d6-075f-4849-88a9-5441edb70777 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:42.018380 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=48 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:42.024265 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e42a89d6-075f-4849-88a9-5441edb70777 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:42.024750 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e42a89d6-075f-4849-88a9-5441edb70777 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:42.025188 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e42a89d6-075f-4849-88a9-5441edb70777 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:42.025882 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e42a89d6-075f-4849-88a9-5441edb70777 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:42.026333 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e42a89d6-075f-4849-88a9-5441edb70777 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:42.026758 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e42a89d6-075f-4849-88a9-5441edb70777 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:42.161318 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e42a89d6-075f-4849-88a9-5441edb70777 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:42.170388 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e42a89d6-075f-4849-88a9-5441edb70777 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.159215 Oct 24 08:55:42.170955 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 261/511] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:42 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:43.195414 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6a95b0c8-ebad-43f3-b168-449ae5173109 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:43.195989 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6a95b0c8-ebad-43f3-b168-449ae5173109 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:43.198529 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=48 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:43.203822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a95b0c8-ebad-43f3-b168-449ae5173109 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:43.204334 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a95b0c8-ebad-43f3-b168-449ae5173109 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:43.204767 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a95b0c8-ebad-43f3-b168-449ae5173109 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:43.205480 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a95b0c8-ebad-43f3-b168-449ae5173109 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:43.205904 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a95b0c8-ebad-43f3-b168-449ae5173109 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:43.206323 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a95b0c8-ebad-43f3-b168-449ae5173109 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:43.344015 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6a95b0c8-ebad-43f3-b168-449ae5173109 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:43.352609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6a95b0c8-ebad-43f3-b168-449ae5173109 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.161951 Oct 24 08:55:43.353181 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 251/512] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:43 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:44.375821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-91569ee7-f6ca-460a-b039-c2eeb92055b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:44.376465 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-91569ee7-f6ca-460a-b039-c2eeb92055b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:44.382944 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91569ee7-f6ca-460a-b039-c2eeb92055b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:44.383430 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91569ee7-f6ca-460a-b039-c2eeb92055b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:44.383874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91569ee7-f6ca-460a-b039-c2eeb92055b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:44.384608 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91569ee7-f6ca-460a-b039-c2eeb92055b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:44.385068 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91569ee7-f6ca-460a-b039-c2eeb92055b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:44.385507 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91569ee7-f6ca-460a-b039-c2eeb92055b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:44.513580 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-91569ee7-f6ca-460a-b039-c2eeb92055b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:44.521634 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-91569ee7-f6ca-460a-b039-c2eeb92055b3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.149662 Oct 24 08:55:44.523413 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 262/513] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:44 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:45.546435 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8276761c-b8c4-4a39-b900-cdcba2cfddde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:45.547029 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8276761c-b8c4-4a39-b900-cdcba2cfddde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:45.553570 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8276761c-b8c4-4a39-b900-cdcba2cfddde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:45.554055 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8276761c-b8c4-4a39-b900-cdcba2cfddde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:45.554490 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8276761c-b8c4-4a39-b900-cdcba2cfddde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:45.555174 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8276761c-b8c4-4a39-b900-cdcba2cfddde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:45.555623 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8276761c-b8c4-4a39-b900-cdcba2cfddde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:45.556044 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8276761c-b8c4-4a39-b900-cdcba2cfddde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:45.687343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8276761c-b8c4-4a39-b900-cdcba2cfddde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:45.696208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8276761c-b8c4-4a39-b900-cdcba2cfddde tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.153592 Oct 24 08:55:45.696781 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 252/514] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:45 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:46.719547 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-981713f1-06a5-4f85-acd5-b9bf919ac27c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:46.720164 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-981713f1-06a5-4f85-acd5-b9bf919ac27c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:46.727134 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-981713f1-06a5-4f85-acd5-b9bf919ac27c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:46.727613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-981713f1-06a5-4f85-acd5-b9bf919ac27c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:46.728053 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-981713f1-06a5-4f85-acd5-b9bf919ac27c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:46.728779 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-981713f1-06a5-4f85-acd5-b9bf919ac27c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:46.729222 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-981713f1-06a5-4f85-acd5-b9bf919ac27c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:46.729942 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-981713f1-06a5-4f85-acd5-b9bf919ac27c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:46.861643 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-981713f1-06a5-4f85-acd5-b9bf919ac27c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:46.870772 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-981713f1-06a5-4f85-acd5-b9bf919ac27c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.155069 Oct 24 08:55:46.871341 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 263/515] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:46 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:47.893707 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e0ad4a22-1283-4634-9ed6-73766c5ccad6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:47.894333 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e0ad4a22-1283-4634-9ed6-73766c5ccad6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:47.901313 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0ad4a22-1283-4634-9ed6-73766c5ccad6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:47.901791 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0ad4a22-1283-4634-9ed6-73766c5ccad6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:47.902240 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0ad4a22-1283-4634-9ed6-73766c5ccad6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:47.902922 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0ad4a22-1283-4634-9ed6-73766c5ccad6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:47.903365 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0ad4a22-1283-4634-9ed6-73766c5ccad6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:47.903779 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0ad4a22-1283-4634-9ed6-73766c5ccad6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:48.034658 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e0ad4a22-1283-4634-9ed6-73766c5ccad6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:48.042922 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e0ad4a22-1283-4634-9ed6-73766c5ccad6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.153066 Oct 24 08:55:48.043489 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 253/516] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:47 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:49.066479 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-3d26165f-2d67-492f-80f7-96e3ed8c50cc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:49.067086 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-3d26165f-2d67-492f-80f7-96e3ed8c50cc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:49.074959 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d26165f-2d67-492f-80f7-96e3ed8c50cc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:49.075439 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d26165f-2d67-492f-80f7-96e3ed8c50cc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:49.075874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d26165f-2d67-492f-80f7-96e3ed8c50cc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:49.076595 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d26165f-2d67-492f-80f7-96e3ed8c50cc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:49.077045 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d26165f-2d67-492f-80f7-96e3ed8c50cc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:49.077473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d26165f-2d67-492f-80f7-96e3ed8c50cc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:49.210334 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-3d26165f-2d67-492f-80f7-96e3ed8c50cc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:49.218511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-3d26165f-2d67-492f-80f7-96e3ed8c50cc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.155932 Oct 24 08:55:49.219074 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 264/517] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:49 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:50.242602 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-395908ae-83f6-48f9-ad7b-89fa9055fec3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:50.243162 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-395908ae-83f6-48f9-ad7b-89fa9055fec3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:50.249562 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-395908ae-83f6-48f9-ad7b-89fa9055fec3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:50.250043 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-395908ae-83f6-48f9-ad7b-89fa9055fec3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:50.250474 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-395908ae-83f6-48f9-ad7b-89fa9055fec3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:50.251160 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-395908ae-83f6-48f9-ad7b-89fa9055fec3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:50.251610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-395908ae-83f6-48f9-ad7b-89fa9055fec3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:50.252036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-395908ae-83f6-48f9-ad7b-89fa9055fec3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:50.269269 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:50.384403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-395908ae-83f6-48f9-ad7b-89fa9055fec3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:50.392590 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-395908ae-83f6-48f9-ad7b-89fa9055fec3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.154295 Oct 24 08:55:50.393167 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 254/518] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:50 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:51.416496 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d2ca0921-41a0-43a7-bd9f-0af3b1703d3c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:51.417107 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d2ca0921-41a0-43a7-bd9f-0af3b1703d3c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:51.423524 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d2ca0921-41a0-43a7-bd9f-0af3b1703d3c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:51.424009 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d2ca0921-41a0-43a7-bd9f-0af3b1703d3c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:51.424484 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d2ca0921-41a0-43a7-bd9f-0af3b1703d3c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:51.425188 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d2ca0921-41a0-43a7-bd9f-0af3b1703d3c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:51.425643 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d2ca0921-41a0-43a7-bd9f-0af3b1703d3c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:51.426072 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d2ca0921-41a0-43a7-bd9f-0af3b1703d3c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:51.610731 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d2ca0921-41a0-43a7-bd9f-0af3b1703d3c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:51.620411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d2ca0921-41a0-43a7-bd9f-0af3b1703d3c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.207769 Oct 24 08:55:51.621000 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 265/519] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:51 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:52.645138 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e8a2c9ad-a649-4d40-93b3-012935cd579c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:52.645858 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e8a2c9ad-a649-4d40-93b3-012935cd579c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:52.653493 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e8a2c9ad-a649-4d40-93b3-012935cd579c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:52.654155 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e8a2c9ad-a649-4d40-93b3-012935cd579c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:52.654488 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e8a2c9ad-a649-4d40-93b3-012935cd579c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:52.655235 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e8a2c9ad-a649-4d40-93b3-012935cd579c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:52.655733 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e8a2c9ad-a649-4d40-93b3-012935cd579c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:52.656208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e8a2c9ad-a649-4d40-93b3-012935cd579c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:52.787876 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e8a2c9ad-a649-4d40-93b3-012935cd579c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:52.796265 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e8a2c9ad-a649-4d40-93b3-012935cd579c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.154911 Oct 24 08:55:52.796862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 255/520] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:52 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:53.819503 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-5071af86-6533-4e57-9534-bfbcd9d64823 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:53.820087 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-5071af86-6533-4e57-9534-bfbcd9d64823 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:53.826407 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5071af86-6533-4e57-9534-bfbcd9d64823 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:53.826883 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5071af86-6533-4e57-9534-bfbcd9d64823 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:53.827316 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5071af86-6533-4e57-9534-bfbcd9d64823 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:53.828002 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5071af86-6533-4e57-9534-bfbcd9d64823 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:53.828476 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5071af86-6533-4e57-9534-bfbcd9d64823 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:53.828902 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5071af86-6533-4e57-9534-bfbcd9d64823 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:53.957963 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-5071af86-6533-4e57-9534-bfbcd9d64823 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:53.967616 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-5071af86-6533-4e57-9534-bfbcd9d64823 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.151972 Oct 24 08:55:53.968223 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 266/521] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:53 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:54.992094 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a39c1e91-07fe-4226-9c24-5700825221a3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:54.992653 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-a39c1e91-07fe-4226-9c24-5700825221a3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:54.999970 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a39c1e91-07fe-4226-9c24-5700825221a3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:55.000495 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a39c1e91-07fe-4226-9c24-5700825221a3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:55.000972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a39c1e91-07fe-4226-9c24-5700825221a3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:55.001676 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a39c1e91-07fe-4226-9c24-5700825221a3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:55.002137 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a39c1e91-07fe-4226-9c24-5700825221a3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:55.002562 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a39c1e91-07fe-4226-9c24-5700825221a3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:55.279723 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-a39c1e91-07fe-4226-9c24-5700825221a3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:55.288214 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-a39c1e91-07fe-4226-9c24-5700825221a3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.300717 Oct 24 08:55:55.288779 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 256/522] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:54 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:56.311453 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-feb8a738-282d-4788-8976-5f67ce7e76fb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:56.311974 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-feb8a738-282d-4788-8976-5f67ce7e76fb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:56.319162 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-feb8a738-282d-4788-8976-5f67ce7e76fb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:56.319726 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-feb8a738-282d-4788-8976-5f67ce7e76fb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:56.319910 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-feb8a738-282d-4788-8976-5f67ce7e76fb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:56.320640 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-feb8a738-282d-4788-8976-5f67ce7e76fb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:56.321099 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-feb8a738-282d-4788-8976-5f67ce7e76fb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:56.321528 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-feb8a738-282d-4788-8976-5f67ce7e76fb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:56.449173 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-feb8a738-282d-4788-8976-5f67ce7e76fb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:56.452555 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:56.458005 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-feb8a738-282d-4788-8976-5f67ce7e76fb tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.150586 Oct 24 08:55:56.458607 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 267/523] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:56 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:57.481505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2c6c4c6f-38ae-4f5f-8214-83c1b0be0a5e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:57.482152 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2c6c4c6f-38ae-4f5f-8214-83c1b0be0a5e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:57.488694 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2c6c4c6f-38ae-4f5f-8214-83c1b0be0a5e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:57.489182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2c6c4c6f-38ae-4f5f-8214-83c1b0be0a5e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:57.489888 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2c6c4c6f-38ae-4f5f-8214-83c1b0be0a5e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:57.490911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2c6c4c6f-38ae-4f5f-8214-83c1b0be0a5e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:57.491179 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2c6c4c6f-38ae-4f5f-8214-83c1b0be0a5e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:57.491611 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2c6c4c6f-38ae-4f5f-8214-83c1b0be0a5e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:57.630558 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2c6c4c6f-38ae-4f5f-8214-83c1b0be0a5e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:57.640080 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2c6c4c6f-38ae-4f5f-8214-83c1b0be0a5e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.162504 Oct 24 08:55:57.640642 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 257/524] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:57 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:58.665110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-54ca2f71-2b41-4a27-bb6a-1ce55d87c382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:58.665724 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-54ca2f71-2b41-4a27-bb6a-1ce55d87c382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:58.673270 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-54ca2f71-2b41-4a27-bb6a-1ce55d87c382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:58.673758 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-54ca2f71-2b41-4a27-bb6a-1ce55d87c382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:58.674204 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-54ca2f71-2b41-4a27-bb6a-1ce55d87c382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:58.674904 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-54ca2f71-2b41-4a27-bb6a-1ce55d87c382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:58.675360 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-54ca2f71-2b41-4a27-bb6a-1ce55d87c382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:58.675789 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-54ca2f71-2b41-4a27-bb6a-1ce55d87c382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:58.813081 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-54ca2f71-2b41-4a27-bb6a-1ce55d87c382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:55:58.821899 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-54ca2f71-2b41-4a27-bb6a-1ce55d87c382 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1736 microversion: 2.1 time: 0.161506 Oct 24 08:55:58.822472 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 268/525] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:58 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1736 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:55:59.846168 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-648cba70-9a8a-40b2-99f4-3661853b34c4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:55:59.846785 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-648cba70-9a8a-40b2-99f4-3661853b34c4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:55:59.853489 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-648cba70-9a8a-40b2-99f4-3661853b34c4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:59.853973 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-648cba70-9a8a-40b2-99f4-3661853b34c4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:59.854438 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-648cba70-9a8a-40b2-99f4-3661853b34c4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:59.855137 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-648cba70-9a8a-40b2-99f4-3661853b34c4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:55:59.855596 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-648cba70-9a8a-40b2-99f4-3661853b34c4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:55:59.856020 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-648cba70-9a8a-40b2-99f4-3661853b34c4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:55:59.993409 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-648cba70-9a8a-40b2-99f4-3661853b34c4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:00.002012 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-648cba70-9a8a-40b2-99f4-3661853b34c4 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1716 microversion: 2.1 time: 0.159695 Oct 24 08:56:00.002658 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 258/526] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:55:59 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1716 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:00.066271 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-59f2b44c-bfbb-47da-a256-6399db40b337 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:00.066787 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-59f2b44c-bfbb-47da-a256-6399db40b337 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:00.074982 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-59f2b44c-bfbb-47da-a256-6399db40b337 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:00.075461 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-59f2b44c-bfbb-47da-a256-6399db40b337 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:00.075896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-59f2b44c-bfbb-47da-a256-6399db40b337 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:00.076622 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-59f2b44c-bfbb-47da-a256-6399db40b337 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:00.077071 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-59f2b44c-bfbb-47da-a256-6399db40b337 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:00.077492 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-59f2b44c-bfbb-47da-a256-6399db40b337 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:00.206005 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-59f2b44c-bfbb-47da-a256-6399db40b337 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] HTTP exception thrown: Cannot 'detach_volume' instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf while it is in vm_state rescued Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-59f2b44c-bfbb-47da-a256-6399db40b337 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Request method failure captured: Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: DELETE /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/os-volume_attachments/c52e4f61-dd63-48b5-a68c-0294cbee81a6 HTTP/1.1 Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlHvWm_27e2NE7FPo6k_ap-1fpqaBaXju_q7yx4ngEwubmyCghP7hRGkUjD3Kh0InrG1jri_Jf3QmFeB_Ar9fw71J3GQum71pn1j3lDf69J9HnlAUxEpDFaBqRZj8Qx4DvfjLqnUdxgNhetuy9MPxaavyu2rUfHlTx9TxAVKDfIab50 Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: fd4b77e78f9d4286aa92f6f90ff8b797 Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-583791902 Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 08:56:00.207613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 08:56:00.209320 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/fd4b77e78f9d4286aa92f6f90ff8b797"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_fd4b77e78f9d4286aa92f6f90ff8b797", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/fd4b77e78f9d4286aa92f6f90ff8b797"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/fd4b77e78f9d4286aa92f6f90ff8b797"}]}] Oct 24 08:56:00.209320 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-ServerRescueNegativeTestJSON-583791902 Oct 24 08:56:00.209320 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: fd4b77e78f9d4286aa92f6f90ff8b797 Oct 24 08:56:00.209320 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-583791902 Oct 24 08:56:00.209320 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-ServerRescueNegativeTestJSON-583791902-project Oct 24 08:56:00.209320 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 08:56:00.209320 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 08:56:00.209320 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: 7127613bde784330bec364f02180291e Oct 24 08:56:00.209320 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-ServerRescueNegativeTestJSON-583791902-project Oct 24 08:56:00.209320 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 08:56:00.209320 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Cannot 'detach_volume' instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf while it is in vm_state rescued Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'server_id': '6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf', 'id': 'c52e4f61-dd63-48b5-a68c-0294cbee81a6'} Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf in vm_state rescued. Cannot detach_volume while the instance is in this state. Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:56:00.210070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:56:00.211844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Oct 24 08:56:00.211844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 24 08:56:00.211844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 24 08:56:00.211844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 24 08:56:00.211844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf while it is in vm_state rescued Oct 24 08:56:00.211844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:56:00.211844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:56:00.211844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:56:00.211844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:56:00.211844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:56:00.211844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:56:00.211844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:56:00.211844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:56:00.211844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf while it is in vm_state rescued Oct 24 08:56:00.211844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:56:00.211844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-59f2b44c-bfbb-47da-a256-6399db40b337 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Returning 409 to user: Cannot 'detach_volume' instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf while it is in vm_state rescued {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:56:00.213096 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-59f2b44c-bfbb-47da-a256-6399db40b337 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "DELETE /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/os-volume_attachments/c52e4f61-dd63-48b5-a68c-0294cbee81a6" status: 409 len: 152 microversion: 2.1 time: 0.147422 Oct 24 08:56:00.213096 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 269/527] 10.222.0.39 () {58 vars in 1416 bytes} [Mon Oct 24 08:56:00 2022] DELETE /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/os-volume_attachments/c52e4f61-dd63-48b5-a68c-0294cbee81a6 => generated 152 bytes in 149 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 24 08:56:00.218788 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-d73b2a9f-ccb8-4480-a55a-bc7a0f4b7bf0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:00.219322 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-d73b2a9f-ccb8-4480-a55a-bc7a0f4b7bf0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:00.226055 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d73b2a9f-ccb8-4480-a55a-bc7a0f4b7bf0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:00.226552 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d73b2a9f-ccb8-4480-a55a-bc7a0f4b7bf0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:00.226989 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d73b2a9f-ccb8-4480-a55a-bc7a0f4b7bf0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:00.227687 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d73b2a9f-ccb8-4480-a55a-bc7a0f4b7bf0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:00.228194 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d73b2a9f-ccb8-4480-a55a-bc7a0f4b7bf0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:00.228592 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d73b2a9f-ccb8-4480-a55a-bc7a0f4b7bf0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:00.296377 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-d73b2a9f-ccb8-4480-a55a-bc7a0f4b7bf0 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "POST /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/action" status: 202 len: 0 microversion: 2.1 time: 0.081201 Oct 24 08:56:00.297131 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 259/528] 10.222.0.39 () {60 vars in 1278 bytes} [Mon Oct 24 08:56:00 2022] POST /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/action => generated 0 bytes in 83 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 08:56:00.306447 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-bbf25f45-f1a3-41fa-a1d6-e151e6cee77b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:00.307190 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-bbf25f45-f1a3-41fa-a1d6-e151e6cee77b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:00.313246 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bbf25f45-f1a3-41fa-a1d6-e151e6cee77b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:00.313814 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bbf25f45-f1a3-41fa-a1d6-e151e6cee77b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:00.314337 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bbf25f45-f1a3-41fa-a1d6-e151e6cee77b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:00.315458 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bbf25f45-f1a3-41fa-a1d6-e151e6cee77b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:00.315976 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bbf25f45-f1a3-41fa-a1d6-e151e6cee77b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:00.316522 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bbf25f45-f1a3-41fa-a1d6-e151e6cee77b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:00.455688 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-bbf25f45-f1a3-41fa-a1d6-e151e6cee77b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:00.465230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-bbf25f45-f1a3-41fa-a1d6-e151e6cee77b tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1724 microversion: 2.1 time: 0.163576 Oct 24 08:56:00.465902 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 270/529] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:00 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1724 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:01.489456 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c5bff08b-76b3-49ce-8c4e-d5074b10f3ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:01.490219 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c5bff08b-76b3-49ce-8c4e-d5074b10f3ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:01.496555 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c5bff08b-76b3-49ce-8c4e-d5074b10f3ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:01.497155 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c5bff08b-76b3-49ce-8c4e-d5074b10f3ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:01.497548 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c5bff08b-76b3-49ce-8c4e-d5074b10f3ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:01.498267 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c5bff08b-76b3-49ce-8c4e-d5074b10f3ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:01.498813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c5bff08b-76b3-49ce-8c4e-d5074b10f3ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:01.499263 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c5bff08b-76b3-49ce-8c4e-d5074b10f3ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:01.631309 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c5bff08b-76b3-49ce-8c4e-d5074b10f3ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:01.639484 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c5bff08b-76b3-49ce-8c4e-d5074b10f3ff tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1724 microversion: 2.1 time: 0.154962 Oct 24 08:56:01.640053 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 260/530] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:01 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1724 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:02.663309 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c14a5928-16af-405e-9f12-ec46ad262615 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:02.663947 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c14a5928-16af-405e-9f12-ec46ad262615 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:02.670359 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c14a5928-16af-405e-9f12-ec46ad262615 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:02.670845 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c14a5928-16af-405e-9f12-ec46ad262615 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:02.671283 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c14a5928-16af-405e-9f12-ec46ad262615 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:02.671996 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c14a5928-16af-405e-9f12-ec46ad262615 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:02.672464 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c14a5928-16af-405e-9f12-ec46ad262615 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:02.672895 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c14a5928-16af-405e-9f12-ec46ad262615 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:02.805741 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c14a5928-16af-405e-9f12-ec46ad262615 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:02.817587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c14a5928-16af-405e-9f12-ec46ad262615 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1724 microversion: 2.1 time: 0.158223 Oct 24 08:56:02.818667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 271/531] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:02 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1724 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:03.842334 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-35d31447-649f-4a2c-873d-c5256ae5ed09 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:03.842968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-35d31447-649f-4a2c-873d-c5256ae5ed09 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:03.849367 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-35d31447-649f-4a2c-873d-c5256ae5ed09 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:03.849878 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-35d31447-649f-4a2c-873d-c5256ae5ed09 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:03.850353 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-35d31447-649f-4a2c-873d-c5256ae5ed09 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:03.851069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-35d31447-649f-4a2c-873d-c5256ae5ed09 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:03.851523 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-35d31447-649f-4a2c-873d-c5256ae5ed09 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:03.851950 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-35d31447-649f-4a2c-873d-c5256ae5ed09 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:03.984220 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-35d31447-649f-4a2c-873d-c5256ae5ed09 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:03.993799 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-35d31447-649f-4a2c-873d-c5256ae5ed09 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1724 microversion: 2.1 time: 0.155236 Oct 24 08:56:03.994159 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 261/532] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:03 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1724 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:05.017270 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8b8d7610-cbbf-4890-a572-479ef72009e3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:05.017886 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8b8d7610-cbbf-4890-a572-479ef72009e3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:05.024608 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8b8d7610-cbbf-4890-a572-479ef72009e3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:05.025123 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8b8d7610-cbbf-4890-a572-479ef72009e3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:05.025566 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8b8d7610-cbbf-4890-a572-479ef72009e3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:05.026264 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8b8d7610-cbbf-4890-a572-479ef72009e3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:05.026723 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8b8d7610-cbbf-4890-a572-479ef72009e3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:05.027178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8b8d7610-cbbf-4890-a572-479ef72009e3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:05.162536 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8b8d7610-cbbf-4890-a572-479ef72009e3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:05.170692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8b8d7610-cbbf-4890-a572-479ef72009e3 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1730 microversion: 2.1 time: 0.157408 Oct 24 08:56:05.171251 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 272/533] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:05 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1730 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:56:05.194543 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c228f5ec-a467-4a84-bf13-66cdde3cfd18 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:05.195117 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c228f5ec-a467-4a84-bf13-66cdde3cfd18 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:05.201546 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c228f5ec-a467-4a84-bf13-66cdde3cfd18 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:05.202161 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c228f5ec-a467-4a84-bf13-66cdde3cfd18 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:05.202453 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c228f5ec-a467-4a84-bf13-66cdde3cfd18 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:05.203151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c228f5ec-a467-4a84-bf13-66cdde3cfd18 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:05.203605 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c228f5ec-a467-4a84-bf13-66cdde3cfd18 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:05.204029 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c228f5ec-a467-4a84-bf13-66cdde3cfd18 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:05.281022 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c228f5ec-a467-4a84-bf13-66cdde3cfd18 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.091012 Oct 24 08:56:05.281621 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 262/534] 10.222.0.39 () {58 vars in 1275 bytes} [Mon Oct 24 08:56:05 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/os-interface => generated 284 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:56:06.158122 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-065e4583-7093-4f5f-b8ab-174e101e3c04 req-e15be58b-75d0-4548-b361-aa2cf39ce7c5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf", "tag": "2a188a19-a722-46ea-96ea-89232fd07436"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:06.169455 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-065e4583-7093-4f5f-b8ab-174e101e3c04 req-e15be58b-75d0-4548-b361-aa2cf39ce7c5 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:06.170065 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-065e4583-7093-4f5f-b8ab-174e101e3c04 req-e15be58b-75d0-4548-b361-aa2cf39ce7c5 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:06.170610 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-065e4583-7093-4f5f-b8ab-174e101e3c04 req-e15be58b-75d0-4548-b361-aa2cf39ce7c5 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:06.182531 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-065e4583-7093-4f5f-b8ab-174e101e3c04 req-e15be58b-75d0-4548-b361-aa2cf39ce7c5 service nova] Creating event network-changed:2a188a19-a722-46ea-96ea-89232fd07436 for instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf on hv-cinder-maste Oct 24 08:56:06.187734 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-065e4583-7093-4f5f-b8ab-174e101e3c04 req-e15be58b-75d0-4548-b361-aa2cf39ce7c5 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033163 Oct 24 08:56:06.188373 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 273/535] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:56:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:56:08.202025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-fb695345-f1b5-407d-8903-e9adcc364745 req-afd19ad1-d205-44d9-af14-ceea7e2abf7e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf", "tag": "2a188a19-a722-46ea-96ea-89232fd07436"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:08.212109 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-fb695345-f1b5-407d-8903-e9adcc364745 req-afd19ad1-d205-44d9-af14-ceea7e2abf7e service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:08.212609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-fb695345-f1b5-407d-8903-e9adcc364745 req-afd19ad1-d205-44d9-af14-ceea7e2abf7e service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:08.213037 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-fb695345-f1b5-407d-8903-e9adcc364745 req-afd19ad1-d205-44d9-af14-ceea7e2abf7e service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:08.224831 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-fb695345-f1b5-407d-8903-e9adcc364745 req-afd19ad1-d205-44d9-af14-ceea7e2abf7e service nova] Creating event network-changed:2a188a19-a722-46ea-96ea-89232fd07436 for instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf on hv-cinder-maste Oct 24 08:56:08.229724 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-fb695345-f1b5-407d-8903-e9adcc364745 req-afd19ad1-d205-44d9-af14-ceea7e2abf7e service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031884 Oct 24 08:56:08.230492 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 263/536] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:56:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:56:25.414774 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=25 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:25.415545 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=32 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:25.476494 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-52cd5d43-2558-4b33-aed9-ac04f4cc457f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:25.477084 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-52cd5d43-2558-4b33-aed9-ac04f4cc457f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:25.483186 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-52cd5d43-2558-4b33-aed9-ac04f4cc457f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:25.483670 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-52cd5d43-2558-4b33-aed9-ac04f4cc457f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:25.484108 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-52cd5d43-2558-4b33-aed9-ac04f4cc457f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:25.484836 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-52cd5d43-2558-4b33-aed9-ac04f4cc457f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:25.485290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-52cd5d43-2558-4b33-aed9-ac04f4cc457f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:25.485714 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-52cd5d43-2558-4b33-aed9-ac04f4cc457f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:25.512690 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-52cd5d43-2558-4b33-aed9-ac04f4cc457f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85/ips" status: 200 len: 151 microversion: 2.1 time: 0.099858 Oct 24 08:56:25.513257 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 274/537] 10.222.0.39 () {58 vars in 1248 bytes} [Mon Oct 24 08:56:25 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85/ips => generated 151 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:56:25.523913 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 24 08:56:25.524454 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Mon Oct 24 08:56:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85/os-volume_attachments (ip 10.222.0.39) !!! Oct 24 08:56:25.525369 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=25 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:25.526112 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=54,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:25.530355 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a9d111b7-e18c-42cb-bd1c-42191df03ee7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c14733e6-7fc6-4b57-b829-55b2304a14e8"}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:25.532863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-a9d111b7-e18c-42cb-bd1c-42191df03ee7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:25.542184 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a9d111b7-e18c-42cb-bd1c-42191df03ee7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:25.542668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a9d111b7-e18c-42cb-bd1c-42191df03ee7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:25.543108 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a9d111b7-e18c-42cb-bd1c-42191df03ee7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:25.543816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a9d111b7-e18c-42cb-bd1c-42191df03ee7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:25.544320 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a9d111b7-e18c-42cb-bd1c-42191df03ee7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:25.544753 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a9d111b7-e18c-42cb-bd1c-42191df03ee7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:25.626262 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-a9d111b7-e18c-42cb-bd1c-42191df03ee7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] HTTP exception thrown: Invalid volume: volume c14733e6-7fc6-4b57-b829-55b2304a14e8 is already attached to instances: a85913b0-03e9-4510-9fc7-1514f34c95c1 Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a9d111b7-e18c-42cb-bd1c-42191df03ee7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Request method failure captured: Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: POST /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85/os-volume_attachments HTTP/1.1 Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 74 Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlH_P2cgqdGxTyt_DjhRKpesrNF98h-yweC7dRlZggOYZXXonk3ROJmezNObJqlbezl2V7Pqsl8fDKOVcqEGBVvIZd3l8-ZBbHliak_eivuyadmAWxfis-R_etsiC3KVVo1LvGlKxFu0GFkQBrTFC5gIfs64iGB92PfDe2tNtbCL85U Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 08:56:25.628007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 08:56:25.629699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b34f10ec378a4622a8f0b0546e84da7f", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}] Oct 24 08:56:25.629699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-AttachVolumeNegativeTest-97597731 Oct 24 08:56:25.629699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 08:56:25.629699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 08:56:25.629699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 08:56:25.629699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 08:56:25.629699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 08:56:25.629699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: ebdef1d69bb14837b9a0c28cbba332fc Oct 24 08:56:25.629699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 08:56:25.629699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Oct 24 08:56:25.629699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {"volumeAttachment": {"volumeId": "c14733e6-7fc6-4b57-b829-55b2304a14e8"}} Oct 24 08:56:25.629699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Invalid volume: volume c14733e6-7fc6-4b57-b829-55b2304a14e8 is already attached to instances: a85913b0-03e9-4510-9fc7-1514f34c95c1 Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'server_id': '8f61c54c-9a6d-445b-8a49-056657208b85', 'body': {'volumeAttachment': {'volumeId': 'c14733e6-7fc6-4b57-b829-55b2304a14e8'}}} Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume c14733e6-7fc6-4b57-b829-55b2304a14e8 is already attached to instances: a85913b0-03e9-4510-9fc7-1514f34c95c1 Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:56:25.630502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume c14733e6-7fc6-4b57-b829-55b2304a14e8 is already attached to instances: a85913b0-03e9-4510-9fc7-1514f34c95c1 Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:56:25.631695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:56:25.632962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:56:25.632962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:56:25.632962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:56:25.632962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume c14733e6-7fc6-4b57-b829-55b2304a14e8 is already attached to instances: a85913b0-03e9-4510-9fc7-1514f34c95c1 Oct 24 08:56:25.632962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:56:25.632962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a9d111b7-e18c-42cb-bd1c-42191df03ee7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Returning 400 to user: Invalid volume: volume c14733e6-7fc6-4b57-b829-55b2304a14e8 is already attached to instances: a85913b0-03e9-4510-9fc7-1514f34c95c1 {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:56:25.632962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-a9d111b7-e18c-42cb-bd1c-42191df03ee7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "POST /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.107310 Oct 24 08:56:25.632962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 264/538] 10.222.0.39 () {60 vars in 1323 bytes} [Mon Oct 24 08:56:25 2022] POST /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85/os-volume_attachments => generated 174 bytes in 109 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 24 08:56:25.638613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-58d689b6-1fd6-4351-8083-e8808d6c53ec tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:25.639109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-58d689b6-1fd6-4351-8083-e8808d6c53ec tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:25.644975 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-58d689b6-1fd6-4351-8083-e8808d6c53ec tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:25.646253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-58d689b6-1fd6-4351-8083-e8808d6c53ec tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:25.646877 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-58d689b6-1fd6-4351-8083-e8808d6c53ec tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:25.647656 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-58d689b6-1fd6-4351-8083-e8808d6c53ec tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:25.648115 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-58d689b6-1fd6-4351-8083-e8808d6c53ec tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:25.648653 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-58d689b6-1fd6-4351-8083-e8808d6c53ec tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:25.674687 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-58d689b6-1fd6-4351-8083-e8808d6c53ec tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Going to try to terminate instance {{(pid=130395) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 08:56:25.749648 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-58d689b6-1fd6-4351-8083-e8808d6c53ec tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "DELETE /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 204 len: 0 microversion: 2.1 time: 0.114479 Oct 24 08:56:25.750323 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 275/539] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 08:56:25 2022] DELETE /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 08:56:25.759785 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-831f6831-ee51-4ab9-89f4-f4c1815dd356 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:25.760845 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-831f6831-ee51-4ab9-89f4-f4c1815dd356 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:25.767684 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-831f6831-ee51-4ab9-89f4-f4c1815dd356 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:25.768255 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-831f6831-ee51-4ab9-89f4-f4c1815dd356 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:25.768698 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-831f6831-ee51-4ab9-89f4-f4c1815dd356 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:25.769401 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-831f6831-ee51-4ab9-89f4-f4c1815dd356 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:25.769857 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-831f6831-ee51-4ab9-89f4-f4c1815dd356 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:25.770284 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-831f6831-ee51-4ab9-89f4-f4c1815dd356 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:25.905534 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-831f6831-ee51-4ab9-89f4-f4c1815dd356 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:25.913250 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-831f6831-ee51-4ab9-89f4-f4c1815dd356 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1684 microversion: 2.1 time: 0.158324 Oct 24 08:56:25.913813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 265/540] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:25 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1684 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:26.937032 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b774ae9f-5280-427f-a29c-4f5534941b3d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:26.937734 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b774ae9f-5280-427f-a29c-4f5534941b3d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:26.944011 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b774ae9f-5280-427f-a29c-4f5534941b3d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:26.944527 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b774ae9f-5280-427f-a29c-4f5534941b3d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:26.944966 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b774ae9f-5280-427f-a29c-4f5534941b3d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:26.945647 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b774ae9f-5280-427f-a29c-4f5534941b3d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:26.946096 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b774ae9f-5280-427f-a29c-4f5534941b3d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:26.946518 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b774ae9f-5280-427f-a29c-4f5534941b3d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:27.077499 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-b774ae9f-5280-427f-a29c-4f5534941b3d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:27.085805 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b774ae9f-5280-427f-a29c-4f5534941b3d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1684 microversion: 2.1 time: 0.153049 Oct 24 08:56:27.086395 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 276/541] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:26 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1684 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:28.109288 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-98f77b9f-f36d-4c70-af2e-e0ce3a3248ac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:28.109892 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-98f77b9f-f36d-4c70-af2e-e0ce3a3248ac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:28.117039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-98f77b9f-f36d-4c70-af2e-e0ce3a3248ac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:28.117533 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-98f77b9f-f36d-4c70-af2e-e0ce3a3248ac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:28.117970 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-98f77b9f-f36d-4c70-af2e-e0ce3a3248ac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:28.118717 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-98f77b9f-f36d-4c70-af2e-e0ce3a3248ac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:28.119180 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-98f77b9f-f36d-4c70-af2e-e0ce3a3248ac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:28.119651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-98f77b9f-f36d-4c70-af2e-e0ce3a3248ac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:28.251873 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-98f77b9f-f36d-4c70-af2e-e0ce3a3248ac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:28.260417 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-98f77b9f-f36d-4c70-af2e-e0ce3a3248ac tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1684 microversion: 2.1 time: 0.154936 Oct 24 08:56:28.260985 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 266/542] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:28 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1684 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:28.838956 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-4533bfc7-d844-464a-ba44-f07bf2b33d5c req-56c84f1c-72f3-4a1d-a6e9-e72df8283901 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8f61c54c-9a6d-445b-8a49-056657208b85", "name": "network-vif-deleted", "tag": "e9bb3baa-6f12-4847-a485-980d16c4f05b"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:28.849526 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-4533bfc7-d844-464a-ba44-f07bf2b33d5c req-56c84f1c-72f3-4a1d-a6e9-e72df8283901 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:28.850324 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-4533bfc7-d844-464a-ba44-f07bf2b33d5c req-56c84f1c-72f3-4a1d-a6e9-e72df8283901 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:28.850856 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-4533bfc7-d844-464a-ba44-f07bf2b33d5c req-56c84f1c-72f3-4a1d-a6e9-e72df8283901 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:28.862634 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-4533bfc7-d844-464a-ba44-f07bf2b33d5c req-56c84f1c-72f3-4a1d-a6e9-e72df8283901 service nova] Creating event network-vif-deleted:e9bb3baa-6f12-4847-a485-980d16c4f05b for instance 8f61c54c-9a6d-445b-8a49-056657208b85 on hv-cinder-maste Oct 24 08:56:28.868654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-4533bfc7-d844-464a-ba44-f07bf2b33d5c req-56c84f1c-72f3-4a1d-a6e9-e72df8283901 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.106794 Oct 24 08:56:28.869911 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 277/543] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:56:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:56:29.284312 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e668e157-dc32-43fa-873b-1b6a287a25d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:29.284836 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e668e157-dc32-43fa-873b-1b6a287a25d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:29.292239 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e668e157-dc32-43fa-873b-1b6a287a25d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:29.292733 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e668e157-dc32-43fa-873b-1b6a287a25d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:29.293532 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e668e157-dc32-43fa-873b-1b6a287a25d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:29.294191 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e668e157-dc32-43fa-873b-1b6a287a25d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:29.294550 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e668e157-dc32-43fa-873b-1b6a287a25d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:29.294980 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e668e157-dc32-43fa-873b-1b6a287a25d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:29.328775 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e668e157-dc32-43fa-873b-1b6a287a25d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:29.329410 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e668e157-dc32-43fa-873b-1b6a287a25d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:29.329914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e668e157-dc32-43fa-873b-1b6a287a25d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:29.330403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.objects.instance [None req-e668e157-dc32-43fa-873b-1b6a287a25d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lazy-loading 'fault' on Instance uuid 8f61c54c-9a6d-445b-8a49-056657208b85 {{(pid=130396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:56:29.380215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e668e157-dc32-43fa-873b-1b6a287a25d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:29.389424 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e668e157-dc32-43fa-873b-1b6a287a25d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 200 len: 1332 microversion: 2.1 time: 0.108953 Oct 24 08:56:29.390014 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 267/544] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:29 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 1332 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:30.415370 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-256f819d-7c51-45ae-aea8-93ef78108437 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:30.416083 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-256f819d-7c51-45ae-aea8-93ef78108437 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:30.422382 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-256f819d-7c51-45ae-aea8-93ef78108437 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:30.422865 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-256f819d-7c51-45ae-aea8-93ef78108437 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:30.423297 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-256f819d-7c51-45ae-aea8-93ef78108437 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:30.423986 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-256f819d-7c51-45ae-aea8-93ef78108437 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:30.424484 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-256f819d-7c51-45ae-aea8-93ef78108437 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:30.424913 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-256f819d-7c51-45ae-aea8-93ef78108437 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:30.438816 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-256f819d-7c51-45ae-aea8-93ef78108437 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] HTTP exception thrown: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-256f819d-7c51-45ae-aea8-93ef78108437 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Request method failure captured: Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 HTTP/1.1 Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlH_P2cgqdGxTyt_DjhRKpesrNF98h-yweC7dRlZggOYZXXonk3ROJmezNObJqlbezl2V7Pqsl8fDKOVcqEGBVvIZd3l8-ZBbHliak_eivuyadmAWxfis-R_etsiC3KVVo1LvGlKxFu0GFkQBrTFC5gIfs64iGB92PfDe2tNtbCL85U Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 08:56:30.440448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 08:56:30.441987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b34f10ec378a4622a8f0b0546e84da7f", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}] Oct 24 08:56:30.441987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-AttachVolumeNegativeTest-97597731 Oct 24 08:56:30.441987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 08:56:30.441987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 08:56:30.441987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 08:56:30.441987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 08:56:30.441987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 08:56:30.441987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: ebdef1d69bb14837b9a0c28cbba332fc Oct 24 08:56:30.441987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 08:56:30.441987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 08:56:30.441987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. Oct 24 08:56:30.441987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': '8f61c54c-9a6d-445b-8a49-056657208b85'} Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:56:30.443110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. Oct 24 08:56:30.444639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:56:30.446040 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-256f819d-7c51-45ae-aea8-93ef78108437 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Returning 404 to user: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:56:30.446040 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-256f819d-7c51-45ae-aea8-93ef78108437 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 404 len: 111 microversion: 2.1 time: 0.031938 Oct 24 08:56:30.446040 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 278/545] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:30 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 08:56:30.451208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5b88caca-b86f-46b7-9268-cbae7e9fdb0d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '.version_select at 0x7fdfeddfe3a0>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:30.477190 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5b88caca-b86f-46b7-9268-cbae7e9fdb0d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1885294353" status: 202 len: 0 microversion: 2.1 time: 0.029879 Oct 24 08:56:30.477769 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 268/546] 10.222.0.39 () {58 vars in 1221 bytes} [Mon Oct 24 08:56:30 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1885294353 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 08:56:31.178064 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f28574e4-04c6-482e-99b0-0910e2266f68 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:31.178550 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f28574e4-04c6-482e-99b0-0910e2266f68 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:31.184451 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f28574e4-04c6-482e-99b0-0910e2266f68 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:31.184924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f28574e4-04c6-482e-99b0-0910e2266f68 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:31.185356 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f28574e4-04c6-482e-99b0-0910e2266f68 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:31.186065 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f28574e4-04c6-482e-99b0-0910e2266f68 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:31.186509 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f28574e4-04c6-482e-99b0-0910e2266f68 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:31.186922 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f28574e4-04c6-482e-99b0-0910e2266f68 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:31.269265 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-f28574e4-04c6-482e-99b0-0910e2266f68 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lazy-loading 'services' on Instance uuid a85913b0-03e9-4510-9fc7-1514f34c95c1 {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:56:31.384777 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f28574e4-04c6-482e-99b0-0910e2266f68 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "DELETE /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1/os-volume_attachments/c14733e6-7fc6-4b57-b829-55b2304a14e8" status: 202 len: 0 microversion: 2.1 time: 0.210985 Oct 24 08:56:31.385357 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 279/547] 10.222.0.39 () {58 vars in 1416 bytes} [Mon Oct 24 08:56:31 2022] DELETE /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1/os-volume_attachments/c14733e6-7fc6-4b57-b829-55b2304a14e8 => generated 0 bytes in 213 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 08:56:31.393736 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b7177bce-8a96-4f81-b3c5-1c02efb06f31 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:31.396574 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b7177bce-8a96-4f81-b3c5-1c02efb06f31 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:31.403838 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b7177bce-8a96-4f81-b3c5-1c02efb06f31 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:31.404523 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b7177bce-8a96-4f81-b3c5-1c02efb06f31 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:31.404964 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b7177bce-8a96-4f81-b3c5-1c02efb06f31 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:31.405663 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b7177bce-8a96-4f81-b3c5-1c02efb06f31 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:31.406115 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b7177bce-8a96-4f81-b3c5-1c02efb06f31 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:31.406550 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b7177bce-8a96-4f81-b3c5-1c02efb06f31 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:31.437340 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b7177bce-8a96-4f81-b3c5-1c02efb06f31 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047444 Oct 24 08:56:31.437976 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 269/548] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 08:56:31 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:56:32.451770 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c9803dd3-b408-469f-9e0f-42285815ad02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:32.454708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c9803dd3-b408-469f-9e0f-42285815ad02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:32.460955 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c9803dd3-b408-469f-9e0f-42285815ad02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:32.461445 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c9803dd3-b408-469f-9e0f-42285815ad02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:32.462094 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c9803dd3-b408-469f-9e0f-42285815ad02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:32.462806 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c9803dd3-b408-469f-9e0f-42285815ad02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:32.463269 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c9803dd3-b408-469f-9e0f-42285815ad02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:32.463708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c9803dd3-b408-469f-9e0f-42285815ad02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:32.496752 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c9803dd3-b408-469f-9e0f-42285815ad02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050276 Oct 24 08:56:32.497323 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 280/549] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 08:56:32 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:56:33.510006 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-081daa18-9e47-4f07-89f0-373cbea29cae tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:33.512246 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-081daa18-9e47-4f07-89f0-373cbea29cae tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:33.518567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-081daa18-9e47-4f07-89f0-373cbea29cae tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:33.519055 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-081daa18-9e47-4f07-89f0-373cbea29cae tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:33.519494 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-081daa18-9e47-4f07-89f0-373cbea29cae tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:33.520845 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-081daa18-9e47-4f07-89f0-373cbea29cae tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:33.521313 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-081daa18-9e47-4f07-89f0-373cbea29cae tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:33.521735 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-081daa18-9e47-4f07-89f0-373cbea29cae tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:33.550075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-081daa18-9e47-4f07-89f0-373cbea29cae tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043980 Oct 24 08:56:33.550654 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 270/550] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 08:56:33 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 24 08:56:33.617510 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1b61e8ed-5290-498c-94f2-5231f847f38e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:33.618041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1b61e8ed-5290-498c-94f2-5231f847f38e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:33.624474 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1b61e8ed-5290-498c-94f2-5231f847f38e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:33.624960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1b61e8ed-5290-498c-94f2-5231f847f38e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:33.625404 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1b61e8ed-5290-498c-94f2-5231f847f38e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:33.626100 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1b61e8ed-5290-498c-94f2-5231f847f38e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:33.626557 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1b61e8ed-5290-498c-94f2-5231f847f38e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:33.626981 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1b61e8ed-5290-498c-94f2-5231f847f38e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:33.652398 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1b61e8ed-5290-498c-94f2-5231f847f38e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Going to try to terminate instance {{(pid=130395) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 08:56:33.727879 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1b61e8ed-5290-498c-94f2-5231f847f38e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "DELETE /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 204 len: 0 microversion: 2.1 time: 0.114991 Oct 24 08:56:33.728546 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 281/551] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 08:56:33 2022] DELETE /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 08:56:33.736932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-36cca95a-a526-4b31-9c51-90ef2c663172 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:33.737545 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-36cca95a-a526-4b31-9c51-90ef2c663172 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:33.744482 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-36cca95a-a526-4b31-9c51-90ef2c663172 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:33.745014 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-36cca95a-a526-4b31-9c51-90ef2c663172 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:33.745487 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-36cca95a-a526-4b31-9c51-90ef2c663172 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:33.746209 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-36cca95a-a526-4b31-9c51-90ef2c663172 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:33.746686 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-36cca95a-a526-4b31-9c51-90ef2c663172 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:33.747137 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-36cca95a-a526-4b31-9c51-90ef2c663172 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:33.884466 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-36cca95a-a526-4b31-9c51-90ef2c663172 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:33.892157 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-36cca95a-a526-4b31-9c51-90ef2c663172 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1681 microversion: 2.1 time: 0.159122 Oct 24 08:56:33.892709 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 271/552] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:33 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1681 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:34.852640 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-28dfeec3-f722-4e59-b417-2de5d49b2a67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:34.853232 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-28dfeec3-f722-4e59-b417-2de5d49b2a67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:34.859078 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-28dfeec3-f722-4e59-b417-2de5d49b2a67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:34.859558 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-28dfeec3-f722-4e59-b417-2de5d49b2a67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:34.859998 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-28dfeec3-f722-4e59-b417-2de5d49b2a67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:34.860717 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-28dfeec3-f722-4e59-b417-2de5d49b2a67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:34.861174 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-28dfeec3-f722-4e59-b417-2de5d49b2a67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:34.861606 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-28dfeec3-f722-4e59-b417-2de5d49b2a67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:34.892706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-28dfeec3-f722-4e59-b417-2de5d49b2a67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/os-volume_attachments/6bbd93af-7386-4dd2-a515-4a6221cff28e" status: 200 len: 194 microversion: 2.20 time: 0.141292 Oct 24 08:56:34.893269 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 282/553] 10.222.0.39 () {60 vars in 1454 bytes} [Mon Oct 24 08:56:34 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/os-volume_attachments/6bbd93af-7386-4dd2-a515-4a6221cff28e => generated 194 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:56:34.915273 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8abef4c7-a440-4ff1-863f-ec1c267207b3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:34.915874 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8abef4c7-a440-4ff1-863f-ec1c267207b3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:34.922691 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8abef4c7-a440-4ff1-863f-ec1c267207b3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:34.923172 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8abef4c7-a440-4ff1-863f-ec1c267207b3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:34.923601 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8abef4c7-a440-4ff1-863f-ec1c267207b3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:34.924322 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8abef4c7-a440-4ff1-863f-ec1c267207b3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:34.924798 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8abef4c7-a440-4ff1-863f-ec1c267207b3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:34.925189 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8abef4c7-a440-4ff1-863f-ec1c267207b3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:34.967480 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-41710d13-253a-43ec-b30d-bb1833793d7d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:34.967973 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-41710d13-253a-43ec-b30d-bb1833793d7d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:34.974078 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-41710d13-253a-43ec-b30d-bb1833793d7d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:34.976985 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-41710d13-253a-43ec-b30d-bb1833793d7d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:34.976985 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-41710d13-253a-43ec-b30d-bb1833793d7d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:34.976985 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-41710d13-253a-43ec-b30d-bb1833793d7d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:34.976985 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-41710d13-253a-43ec-b30d-bb1833793d7d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:34.977345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-41710d13-253a-43ec-b30d-bb1833793d7d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:35.056722 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8abef4c7-a440-4ff1-863f-ec1c267207b3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:35.058850 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-41710d13-253a-43ec-b30d-bb1833793d7d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lazy-loading 'services' on Instance uuid 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:56:35.064488 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8abef4c7-a440-4ff1-863f-ec1c267207b3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1681 microversion: 2.1 time: 0.153188 Oct 24 08:56:35.065065 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 272/554] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:34 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1681 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:35.134860 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-41710d13-253a-43ec-b30d-bb1833793d7d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "DELETE /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/os-volume_attachments/6bbd93af-7386-4dd2-a515-4a6221cff28e" status: 202 len: 0 microversion: 2.20 time: 0.171235 Oct 24 08:56:35.135562 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 283/555] 10.222.0.39 () {60 vars in 1457 bytes} [Mon Oct 24 08:56:34 2022] DELETE /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/os-volume_attachments/6bbd93af-7386-4dd2-a515-4a6221cff28e => generated 0 bytes in 173 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 24 08:56:35.144343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-1072ef1c-638e-4ac3-839b-e0914d330079 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:35.147100 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-1072ef1c-638e-4ac3-839b-e0914d330079 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:35.153117 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1072ef1c-638e-4ac3-839b-e0914d330079 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:35.153596 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1072ef1c-638e-4ac3-839b-e0914d330079 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:35.154023 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1072ef1c-638e-4ac3-839b-e0914d330079 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:35.154713 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1072ef1c-638e-4ac3-839b-e0914d330079 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:35.155159 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1072ef1c-638e-4ac3-839b-e0914d330079 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:35.155568 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1072ef1c-638e-4ac3-839b-e0914d330079 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:35.193943 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-1072ef1c-638e-4ac3-839b-e0914d330079 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047560 Oct 24 08:56:35.194928 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 273/556] 10.222.0.39 () {60 vars in 1343 bytes} [Mon Oct 24 08:56:35 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:56:36.089922 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-15ce2894-833f-4df2-be5f-d9920409dcc2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:36.090779 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-15ce2894-833f-4df2-be5f-d9920409dcc2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:36.100200 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-15ce2894-833f-4df2-be5f-d9920409dcc2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:36.100916 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-15ce2894-833f-4df2-be5f-d9920409dcc2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:36.100916 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-15ce2894-833f-4df2-be5f-d9920409dcc2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:36.101835 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-15ce2894-833f-4df2-be5f-d9920409dcc2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:36.102337 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-15ce2894-833f-4df2-be5f-d9920409dcc2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:36.102768 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-15ce2894-833f-4df2-be5f-d9920409dcc2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:36.208107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-533a2853-c279-4035-a826-d6594ba70d62 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:36.211392 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-533a2853-c279-4035-a826-d6594ba70d62 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:36.218495 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-533a2853-c279-4035-a826-d6594ba70d62 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:36.219190 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-533a2853-c279-4035-a826-d6594ba70d62 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:36.219803 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-533a2853-c279-4035-a826-d6594ba70d62 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:36.221174 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-533a2853-c279-4035-a826-d6594ba70d62 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:36.221899 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-533a2853-c279-4035-a826-d6594ba70d62 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:36.222497 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-533a2853-c279-4035-a826-d6594ba70d62 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:36.247301 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-15ce2894-833f-4df2-be5f-d9920409dcc2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:36.257166 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-15ce2894-833f-4df2-be5f-d9920409dcc2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1681 microversion: 2.1 time: 0.171416 Oct 24 08:56:36.258254 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 284/557] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:36 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1681 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:36.269242 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-533a2853-c279-4035-a826-d6594ba70d62 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.065760 Oct 24 08:56:36.270858 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 274/558] 10.222.0.39 () {60 vars in 1343 bytes} [Mon Oct 24 08:56:36 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:56:37.125369 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-6314fb5a-17bf-452b-a9d6-dd70e6e2da45 req-c4ff4b1f-0aa8-44b3-baa2-c934377bfbb8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a85913b0-03e9-4510-9fc7-1514f34c95c1", "name": "network-vif-deleted", "tag": "91e7f21e-e0a0-4b2a-9221-473c98847085"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:37.137311 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-6314fb5a-17bf-452b-a9d6-dd70e6e2da45 req-c4ff4b1f-0aa8-44b3-baa2-c934377bfbb8 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:37.137895 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-6314fb5a-17bf-452b-a9d6-dd70e6e2da45 req-c4ff4b1f-0aa8-44b3-baa2-c934377bfbb8 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:37.138408 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-6314fb5a-17bf-452b-a9d6-dd70e6e2da45 req-c4ff4b1f-0aa8-44b3-baa2-c934377bfbb8 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:37.153452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-6314fb5a-17bf-452b-a9d6-dd70e6e2da45 req-c4ff4b1f-0aa8-44b3-baa2-c934377bfbb8 service nova] Creating event network-vif-deleted:91e7f21e-e0a0-4b2a-9221-473c98847085 for instance a85913b0-03e9-4510-9fc7-1514f34c95c1 on hv-cinder-maste Oct 24 08:56:37.158556 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-6314fb5a-17bf-452b-a9d6-dd70e6e2da45 req-c4ff4b1f-0aa8-44b3-baa2-c934377bfbb8 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036903 Oct 24 08:56:37.159249 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 285/559] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:56:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:56:37.283189 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6b9d851d-8568-4148-bc76-8378bcf35ac6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:37.285477 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6b9d851d-8568-4148-bc76-8378bcf35ac6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:37.286883 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8c919c79-a5ba-4ed8-a53e-7777f90239f9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:37.287342 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8c919c79-a5ba-4ed8-a53e-7777f90239f9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:37.298413 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b9d851d-8568-4148-bc76-8378bcf35ac6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:37.298815 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c919c79-a5ba-4ed8-a53e-7777f90239f9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:37.298967 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b9d851d-8568-4148-bc76-8378bcf35ac6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:37.299428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b9d851d-8568-4148-bc76-8378bcf35ac6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:37.299599 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c919c79-a5ba-4ed8-a53e-7777f90239f9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:37.300033 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b9d851d-8568-4148-bc76-8378bcf35ac6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:37.300197 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c919c79-a5ba-4ed8-a53e-7777f90239f9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:37.300906 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c919c79-a5ba-4ed8-a53e-7777f90239f9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:37.302176 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c919c79-a5ba-4ed8-a53e-7777f90239f9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:37.302895 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c919c79-a5ba-4ed8-a53e-7777f90239f9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:37.304191 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b9d851d-8568-4148-bc76-8378bcf35ac6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:37.304650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b9d851d-8568-4148-bc76-8378bcf35ac6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:37.352107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6b9d851d-8568-4148-bc76-8378bcf35ac6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.072541 Oct 24 08:56:37.353222 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 275/560] 10.222.0.39 () {60 vars in 1343 bytes} [Mon Oct 24 08:56:37 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6/os-volume_attachments => generated 25 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:37.354014 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c919c79-a5ba-4ed8-a53e-7777f90239f9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:37.354657 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c919c79-a5ba-4ed8-a53e-7777f90239f9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:37.355232 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c919c79-a5ba-4ed8-a53e-7777f90239f9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:37.355847 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-8c919c79-a5ba-4ed8-a53e-7777f90239f9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lazy-loading 'fault' on Instance uuid a85913b0-03e9-4510-9fc7-1514f34c95c1 {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:56:37.425049 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8c919c79-a5ba-4ed8-a53e-7777f90239f9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:37.434026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8c919c79-a5ba-4ed8-a53e-7777f90239f9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1332 microversion: 2.1 time: 0.151806 Oct 24 08:56:37.434773 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 286/561] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:37 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1332 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:37.449223 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-833ba6e3-fb59-402b-94ac-76450c905eb2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:37.449657 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-833ba6e3-fb59-402b-94ac-76450c905eb2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:37.458376 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-833ba6e3-fb59-402b-94ac-76450c905eb2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:37.458969 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-833ba6e3-fb59-402b-94ac-76450c905eb2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:37.459309 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-833ba6e3-fb59-402b-94ac-76450c905eb2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:37.460013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-833ba6e3-fb59-402b-94ac-76450c905eb2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:37.460493 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-833ba6e3-fb59-402b-94ac-76450c905eb2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:37.460950 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-833ba6e3-fb59-402b-94ac-76450c905eb2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:37.488724 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-833ba6e3-fb59-402b-94ac-76450c905eb2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Going to try to terminate instance {{(pid=130396) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 08:56:37.582122 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-833ba6e3-fb59-402b-94ac-76450c905eb2 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "DELETE /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 204 len: 0 microversion: 2.20 time: 0.151269 Oct 24 08:56:37.584473 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 276/562] 10.222.0.39 () {60 vars in 1280 bytes} [Mon Oct 24 08:56:37 2022] DELETE /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 24 08:56:37.593258 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-613618fc-b576-4a46-b0c1-81b2ff46c4a6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:37.593808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-613618fc-b576-4a46-b0c1-81b2ff46c4a6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:37.600991 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-613618fc-b576-4a46-b0c1-81b2ff46c4a6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:37.601470 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-613618fc-b576-4a46-b0c1-81b2ff46c4a6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:37.601903 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-613618fc-b576-4a46-b0c1-81b2ff46c4a6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:37.602595 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-613618fc-b576-4a46-b0c1-81b2ff46c4a6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:37.603045 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-613618fc-b576-4a46-b0c1-81b2ff46c4a6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:37.603461 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-613618fc-b576-4a46-b0c1-81b2ff46c4a6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:37.733349 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-613618fc-b576-4a46-b0c1-81b2ff46c4a6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:37.737058 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=2,nova_cell1:INSERT=4 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:37.743523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-613618fc-b576-4a46-b0c1-81b2ff46c4a6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:37.744998 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-613618fc-b576-4a46-b0c1-81b2ff46c4a6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:37.746167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-613618fc-b576-4a46-b0c1-81b2ff46c4a6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1723 microversion: 2.20 time: 0.157139 Oct 24 08:56:37.746728 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 287/563] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:56:37 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1723 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:56:37.992171 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-66abe782-2e20-4b72-8510-e62be27e5ed9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:37.993010 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-66abe782-2e20-4b72-8510-e62be27e5ed9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:37.999208 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-66abe782-2e20-4b72-8510-e62be27e5ed9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:37.999700 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-66abe782-2e20-4b72-8510-e62be27e5ed9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:38.000162 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-66abe782-2e20-4b72-8510-e62be27e5ed9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:38.000958 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-66abe782-2e20-4b72-8510-e62be27e5ed9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:38.001411 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-66abe782-2e20-4b72-8510-e62be27e5ed9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:38.001837 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-66abe782-2e20-4b72-8510-e62be27e5ed9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:38.129681 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.objects.instance [None req-66abe782-2e20-4b72-8510-e62be27e5ed9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lazy-loading 'services' on Instance uuid 89a2295a-90e9-4fc1-be3b-a130f7e71528 {{(pid=130396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:56:38.250648 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-66abe782-2e20-4b72-8510-e62be27e5ed9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "DELETE /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/os-volume_attachments/44952977-e924-4e63-a390-c688647bdaba" status: 202 len: 0 microversion: 2.1 time: 0.263659 Oct 24 08:56:38.251359 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 277/564] 10.222.0.39 () {58 vars in 1416 bytes} [Mon Oct 24 08:56:37 2022] DELETE /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/os-volume_attachments/44952977-e924-4e63-a390-c688647bdaba => generated 0 bytes in 265 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 08:56:38.460562 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c097d7e5-5ed6-4d5d-8488-747e7880d568 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:38.461094 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c097d7e5-5ed6-4d5d-8488-747e7880d568 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:38.468832 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c097d7e5-5ed6-4d5d-8488-747e7880d568 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:38.469309 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c097d7e5-5ed6-4d5d-8488-747e7880d568 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:38.469741 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c097d7e5-5ed6-4d5d-8488-747e7880d568 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:38.470438 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c097d7e5-5ed6-4d5d-8488-747e7880d568 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:38.470884 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c097d7e5-5ed6-4d5d-8488-747e7880d568 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:38.471301 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c097d7e5-5ed6-4d5d-8488-747e7880d568 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:38.503404 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c097d7e5-5ed6-4d5d-8488-747e7880d568 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:38.503871 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c097d7e5-5ed6-4d5d-8488-747e7880d568 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:38.504873 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c097d7e5-5ed6-4d5d-8488-747e7880d568 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:38.505708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-c097d7e5-5ed6-4d5d-8488-747e7880d568 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lazy-loading 'fault' on Instance uuid a85913b0-03e9-4510-9fc7-1514f34c95c1 {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:56:38.546430 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c097d7e5-5ed6-4d5d-8488-747e7880d568 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:38.554186 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c097d7e5-5ed6-4d5d-8488-747e7880d568 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 200 len: 1332 microversion: 2.1 time: 0.098129 Oct 24 08:56:38.554747 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 288/565] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:38 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 1332 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:38.774470 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7e85811b-6248-4728-9c0e-a54a2e7c156c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:38.775067 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7e85811b-6248-4728-9c0e-a54a2e7c156c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:38.781260 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7e85811b-6248-4728-9c0e-a54a2e7c156c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:38.781728 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7e85811b-6248-4728-9c0e-a54a2e7c156c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:38.782160 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7e85811b-6248-4728-9c0e-a54a2e7c156c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:38.782834 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7e85811b-6248-4728-9c0e-a54a2e7c156c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:38.783279 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7e85811b-6248-4728-9c0e-a54a2e7c156c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:38.783696 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7e85811b-6248-4728-9c0e-a54a2e7c156c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:38.917741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7e85811b-6248-4728-9c0e-a54a2e7c156c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:38.921223 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:38.927283 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7e85811b-6248-4728-9c0e-a54a2e7c156c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:38.928763 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7e85811b-6248-4728-9c0e-a54a2e7c156c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:38.929918 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7e85811b-6248-4728-9c0e-a54a2e7c156c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1723 microversion: 2.20 time: 0.161346 Oct 24 08:56:38.930489 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 278/566] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:56:38 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1723 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:56:39.577992 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-acc425e8-9d54-4d47-9a23-35f3ca44c434 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:39.578613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-acc425e8-9d54-4d47-9a23-35f3ca44c434 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:39.588249 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-acc425e8-9d54-4d47-9a23-35f3ca44c434 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:39.588758 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-acc425e8-9d54-4d47-9a23-35f3ca44c434 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:39.589270 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-acc425e8-9d54-4d47-9a23-35f3ca44c434 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:39.590121 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-acc425e8-9d54-4d47-9a23-35f3ca44c434 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:39.590619 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-acc425e8-9d54-4d47-9a23-35f3ca44c434 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:39.591116 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-acc425e8-9d54-4d47-9a23-35f3ca44c434 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:39.606214 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-acc425e8-9d54-4d47-9a23-35f3ca44c434 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] HTTP exception thrown: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-acc425e8-9d54-4d47-9a23-35f3ca44c434 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Request method failure captured: Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 HTTP/1.1 Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlH_P2cgqdGxTyt_DjhRKpesrNF98h-yweC7dRlZggOYZXXonk3ROJmezNObJqlbezl2V7Pqsl8fDKOVcqEGBVvIZd3l8-ZBbHliak_eivuyadmAWxfis-R_etsiC3KVVo1LvGlKxFu0GFkQBrTFC5gIfs64iGB92PfDe2tNtbCL85U Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 08:56:39.607840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 08:56:39.609422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b34f10ec378a4622a8f0b0546e84da7f", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}] Oct 24 08:56:39.609422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-AttachVolumeNegativeTest-97597731 Oct 24 08:56:39.609422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 08:56:39.609422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 08:56:39.609422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 08:56:39.609422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 08:56:39.609422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 08:56:39.609422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: ebdef1d69bb14837b9a0c28cbba332fc Oct 24 08:56:39.609422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 08:56:39.609422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 08:56:39.609422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. Oct 24 08:56:39.609422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': 'a85913b0-03e9-4510-9fc7-1514f34c95c1'} Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:56:39.610353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. Oct 24 08:56:39.611808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:56:39.613128 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-acc425e8-9d54-4d47-9a23-35f3ca44c434 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Returning 404 to user: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:56:39.613128 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-acc425e8-9d54-4d47-9a23-35f3ca44c434 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 404 len: 111 microversion: 2.1 time: 0.035698 Oct 24 08:56:39.613128 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 289/567] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:39 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 08:56:39.618869 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e309b998-1825-4295-af35-01147eaaf8e2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '.version_select at 0x7fdfedd5fe50>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:39.641720 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e309b998-1825-4295-af35-01147eaaf8e2 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1141887983" status: 202 len: 0 microversion: 2.1 time: 0.026499 Oct 24 08:56:39.642297 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 279/568] 10.222.0.39 () {58 vars in 1221 bytes} [Mon Oct 24 08:56:39 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1141887983 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 08:56:39.955122 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7e08e58b-660b-4914-ad13-6488122557a7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:39.956055 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7e08e58b-660b-4914-ad13-6488122557a7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:39.963051 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e08e58b-660b-4914-ad13-6488122557a7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:39.963954 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e08e58b-660b-4914-ad13-6488122557a7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:39.964649 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e08e58b-660b-4914-ad13-6488122557a7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:39.965870 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e08e58b-660b-4914-ad13-6488122557a7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:39.966621 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e08e58b-660b-4914-ad13-6488122557a7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:39.967347 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e08e58b-660b-4914-ad13-6488122557a7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:40.472521 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4f88678e-8a7f-4a12-9e94-1d110a4188f9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:40.477799 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7e08e58b-660b-4914-ad13-6488122557a7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:40.485583 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4f88678e-8a7f-4a12-9e94-1d110a4188f9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:40.486110 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4f88678e-8a7f-4a12-9e94-1d110a4188f9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:40.486551 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4f88678e-8a7f-4a12-9e94-1d110a4188f9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:40.492738 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7e08e58b-660b-4914-ad13-6488122557a7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:40.495338 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7e08e58b-660b-4914-ad13-6488122557a7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:40.497224 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7e08e58b-660b-4914-ad13-6488122557a7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1723 microversion: 2.20 time: 0.546001 Oct 24 08:56:40.498665 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 290/569] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:56:39 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1723 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:56:40.564709 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4f88678e-8a7f-4a12-9e94-1d110a4188f9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Going to try to stop instance {{(pid=130396) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 24 08:56:40.618595 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4f88678e-8a7f-4a12-9e94-1d110a4188f9 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "POST /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/action" status: 202 len: 0 microversion: 2.1 time: 0.157465 Oct 24 08:56:40.619209 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 280/570] 10.222.0.39 () {60 vars in 1278 bytes} [Mon Oct 24 08:56:40 2022] POST /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/action => generated 0 bytes in 159 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 24 08:56:40.628462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-329b51ec-615b-462f-8683-2242e0c226f6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:40.638368 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-329b51ec-615b-462f-8683-2242e0c226f6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:40.650350 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-329b51ec-615b-462f-8683-2242e0c226f6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:40.651284 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-329b51ec-615b-462f-8683-2242e0c226f6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:40.652006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-329b51ec-615b-462f-8683-2242e0c226f6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:40.655773 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-329b51ec-615b-462f-8683-2242e0c226f6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:40.656451 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-329b51ec-615b-462f-8683-2242e0c226f6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:40.656974 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-329b51ec-615b-462f-8683-2242e0c226f6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:40.685336 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-bf780de9-3f34-4269-bf47-a5b3ebad25c4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Action: 'create', calling method: .version_select at 0x7fdfedeb3c10>, body: {"keypair": {"name": "tempest-keypair-1846818907", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK2IAwBzSc/iLgwHrMSo3Hun0etI4XR477ki7M12KgqBmKQSI7oGvyhcxLyJZBzRswMQkmPi3iULfEUYgJi7QTTcIYsxx0pt4Wy/Tf1Se40TLUu/Ygja3mKHWs/qcReKRw=="}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:40.702452 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-bf780de9-3f34-4269-bf47-a5b3ebad25c4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Getting quotas for project b34f10ec378a4622a8f0b0546e84da7f. Resources: {'key_pairs'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:56:40.706745 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-bf780de9-3f34-4269-bf47-a5b3ebad25c4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Getting quotas for user ebdef1d69bb14837b9a0c28cbba332fc and project b34f10ec378a4622a8f0b0546e84da7f. Resources: {'key_pairs'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:56:40.731048 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-bf780de9-3f34-4269-bf47-a5b3ebad25c4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.050953 Oct 24 08:56:40.731653 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 281/571] 10.222.0.39 () {60 vars in 1159 bytes} [Mon Oct 24 08:56:40 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:56:40.839661 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-329b51ec-615b-462f-8683-2242e0c226f6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:40.851688 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-329b51ec-615b-462f-8683-2242e0c226f6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1678 microversion: 2.1 time: 0.226654 Oct 24 08:56:40.861704 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 291/572] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:40 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1678 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:40.885044 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-f2fd8a5d-e559-40a1-8269-9732a0c99335 req-3d609e99-ed01-40a9-8fe4-8700d5f61ddd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "71e106ba-ad54-4e9c-8e42-9477c8db8cf6", "name": "network-vif-deleted", "tag": "b8754c97-b769-418c-b461-714a5809d621"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:40.896879 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-f2fd8a5d-e559-40a1-8269-9732a0c99335 req-3d609e99-ed01-40a9-8fe4-8700d5f61ddd service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:40.897391 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-f2fd8a5d-e559-40a1-8269-9732a0c99335 req-3d609e99-ed01-40a9-8fe4-8700d5f61ddd service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:40.897833 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-f2fd8a5d-e559-40a1-8269-9732a0c99335 req-3d609e99-ed01-40a9-8fe4-8700d5f61ddd service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:40.916912 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-f2fd8a5d-e559-40a1-8269-9732a0c99335 req-3d609e99-ed01-40a9-8fe4-8700d5f61ddd service nova] Creating event network-vif-deleted:b8754c97-b769-418c-b461-714a5809d621 for instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 on hv-cinder-maste Oct 24 08:56:40.921991 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-f2fd8a5d-e559-40a1-8269-9732a0c99335 req-3d609e99-ed01-40a9-8fe4-8700d5f61ddd service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041160 Oct 24 08:56:40.923956 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 282/573] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:56:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:56:41.527591 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-50364697-779f-4c27-a88e-3af1e10961fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:41.528342 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-50364697-779f-4c27-a88e-3af1e10961fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:41.536414 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-50364697-779f-4c27-a88e-3af1e10961fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:41.536682 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-50364697-779f-4c27-a88e-3af1e10961fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:41.537188 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-50364697-779f-4c27-a88e-3af1e10961fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:41.538051 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-50364697-779f-4c27-a88e-3af1e10961fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:41.538692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-50364697-779f-4c27-a88e-3af1e10961fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:41.540513 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-50364697-779f-4c27-a88e-3af1e10961fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:41.573931 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-50364697-779f-4c27-a88e-3af1e10961fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:41.574548 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-50364697-779f-4c27-a88e-3af1e10961fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:41.575061 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-50364697-779f-4c27-a88e-3af1e10961fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:41.575584 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-50364697-779f-4c27-a88e-3af1e10961fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lazy-loading 'fault' on Instance uuid 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:56:41.627887 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-50364697-779f-4c27-a88e-3af1e10961fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:41.639822 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-50364697-779f-4c27-a88e-3af1e10961fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:41.642299 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-50364697-779f-4c27-a88e-3af1e10961fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:41.643685 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-50364697-779f-4c27-a88e-3af1e10961fe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 200 len: 1370 microversion: 2.20 time: 0.120038 Oct 24 08:56:41.644777 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 292/574] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:56:41 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 1370 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:56:41.879006 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-60d4f774-42d6-4c87-b3c2-467771b112b1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:41.879752 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-60d4f774-42d6-4c87-b3c2-467771b112b1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:41.886512 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-60d4f774-42d6-4c87-b3c2-467771b112b1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:41.887180 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-60d4f774-42d6-4c87-b3c2-467771b112b1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:41.887755 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-60d4f774-42d6-4c87-b3c2-467771b112b1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:41.888657 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-60d4f774-42d6-4c87-b3c2-467771b112b1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:41.889249 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-60d4f774-42d6-4c87-b3c2-467771b112b1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:41.890080 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-60d4f774-42d6-4c87-b3c2-467771b112b1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:42.104839 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-60d4f774-42d6-4c87-b3c2-467771b112b1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:42.112849 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-60d4f774-42d6-4c87-b3c2-467771b112b1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1678 microversion: 2.1 time: 0.239327 Oct 24 08:56:42.114442 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 283/575] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:41 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1678 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:42.300061 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1090183718", "imageRef": "bba331ba-faf5-4a56-81d9-6e84e9609665", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "151b561f-53f8-4019-8ed3-b08a9e67fe57"}], "security_groups": [{"name": "tempest-securitygroup--2147178119"}], "key_name": "tempest-keypair-1846818907", "user_data": "}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:42.407149 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.network.neutron [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] validate_networks() for [('151b561f-53f8-4019-8ed3-b08a9e67fe57', None, None, None, None, None)] {{(pid=130395) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 08:56:42.671042 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6aef2250-9665-456d-84ba-cd39b0735002 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:42.671574 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6aef2250-9665-456d-84ba-cd39b0735002 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:42.678125 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6aef2250-9665-456d-84ba-cd39b0735002 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:42.678622 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6aef2250-9665-456d-84ba-cd39b0735002 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:42.679066 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6aef2250-9665-456d-84ba-cd39b0735002 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:42.679774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6aef2250-9665-456d-84ba-cd39b0735002 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:42.680257 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6aef2250-9665-456d-84ba-cd39b0735002 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:42.680688 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6aef2250-9665-456d-84ba-cd39b0735002 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:42.694767 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-6aef2250-9665-456d-84ba-cd39b0735002 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] HTTP exception thrown: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6aef2250-9665-456d-84ba-cd39b0735002 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Request method failure captured: Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 HTTP/1.1 Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlH7mBC-BQLg0H8yK7G2g-HOPUEsdjQL5a9CNStTzz2vRIXjmn3BKvY3yRoT_VlpqjhmiOscl52jQCwEMfyH6tF1HUFCQtCOEAYK8cH5PiF_zTNaUGNGpBjSQoGhunfKNz-_3ckGEvv273V4IyJwd9U0Vvic8VBlr37q6YrJuFn9p-Y Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Openstack-Nova-Api-Version: 2.20 Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 7ff48b26c576495f8bb8c88d20d246cc Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 08:56:42.696458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 08:56:42.698151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/7ff48b26c576495f8bb8c88d20d246cc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_7ff48b26c576495f8bb8c88d20d246cc", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/7ff48b26c576495f8bb8c88d20d246cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/7ff48b26c576495f8bb8c88d20d246cc"}]}] Oct 24 08:56:42.698151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 08:56:42.698151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 7ff48b26c576495f8bb8c88d20d246cc Oct 24 08:56:42.698151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 08:56:42.698151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-AttachVolumeShelveTestJSON-160255499-project Oct 24 08:56:42.698151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 08:56:42.698151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 08:56:42.698151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 552c23b8456a40ec8e01297aca7965d3 Oct 24 08:56:42.698151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-AttachVolumeShelveTestJSON-160255499-project Oct 24 08:56:42.698151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 08:56:42.698151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. Oct 24 08:56:42.698151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': '71e106ba-ad54-4e9c-8e42-9477c8db8cf6'} Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:56:42.699428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. Oct 24 08:56:42.701816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:56:42.703695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6aef2250-9665-456d-84ba-cd39b0735002 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Returning 404 to user: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:56:42.703695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6aef2250-9665-456d-84ba-cd39b0735002 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 404 len: 111 microversion: 2.20 time: 0.030904 Oct 24 08:56:42.703695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 284/576] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:56:42 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 24 08:56:42.706569 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a7482964-2db3-4f0b-92b0-9c52b745ea2e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '.version_select at 0x7fdfee1614c0>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:42.726017 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-a7482964-2db3-4f0b-92b0-9c52b745ea2e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-156035562" status: 204 len: 0 microversion: 2.20 time: 0.022645 Oct 24 08:56:42.726666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 285/577] 10.222.0.39 () {60 vars in 1259 bytes} [Mon Oct 24 08:56:42 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-156035562 => generated 0 bytes in 24 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 24 08:56:42.836710 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Flavor limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 08:56:42.837183 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Image limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 08:56:42.837620 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Flavor pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 08:56:42.838039 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Image pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 08:56:42.838456 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 08:56:42.847246 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:42.847730 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:42.848257 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:42.849062 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:42.849524 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:42.849976 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:42.867705 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Getting quotas for project b34f10ec378a4622a8f0b0546e84da7f. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:56:42.872317 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Getting quotas for user ebdef1d69bb14837b9a0c28cbba332fc and project b34f10ec378a4622a8f0b0546e84da7f. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:56:42.885484 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Going to run 1 instances... {{(pid=130395) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 08:56:42.887816 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bba331ba-faf5-4a56-81d9-6e84e9609665',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130395) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 08:56:42.942395 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-bf1c29bc-0c39-42e7-9ceb-e43777506d94 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.645774 Oct 24 08:56:42.943497 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 293/578] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 08:56:42 2022] POST /compute/v2.1/servers => generated 402 bytes in 648 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 08:56:42.954828 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9dda8785-fcfa-4947-bf13-bf42b41aabf5 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:42.955440 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9dda8785-fcfa-4947-bf13-bf42b41aabf5 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:43.045890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9dda8785-fcfa-4947-bf13-bf42b41aabf5 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:43.054041 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9dda8785-fcfa-4947-bf13-bf42b41aabf5 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 200 len: 1242 microversion: 2.1 time: 0.103396 Oct 24 08:56:43.055277 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 286/579] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:42 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 1242 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:43.139057 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-03f94fe1-8a5a-4a99-960d-f7dbc3dc595e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:43.139609 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-03f94fe1-8a5a-4a99-960d-f7dbc3dc595e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:43.146843 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-03f94fe1-8a5a-4a99-960d-f7dbc3dc595e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:43.147426 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-03f94fe1-8a5a-4a99-960d-f7dbc3dc595e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:43.147838 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-03f94fe1-8a5a-4a99-960d-f7dbc3dc595e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:43.148471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-03f94fe1-8a5a-4a99-960d-f7dbc3dc595e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:43.149089 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-03f94fe1-8a5a-4a99-960d-f7dbc3dc595e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:43.149532 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-03f94fe1-8a5a-4a99-960d-f7dbc3dc595e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:43.375161 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-03f94fe1-8a5a-4a99-960d-f7dbc3dc595e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:43.385552 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-03f94fe1-8a5a-4a99-960d-f7dbc3dc595e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1678 microversion: 2.1 time: 0.250427 Oct 24 08:56:43.386510 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 294/580] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:43 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1678 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:43.543675 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-46153d49-5345-4a96-bca4-0478f5e139c5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Action: 'create', calling method: .version_select at 0x7fdfeddf3a60>, body: {"keypair": {"name": "tempest-keypair-492103469", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAep4sby8kBbGQzKVm7WcgHhsjkRzAA87o7HIqgNlj6gJJvhE56IJHeywYbE5vZY/WhxNqIxSGwr3D9oyGrpx17aO1lRhO1z1F9Qkr8RPkWLYHL1n9IdTIuzrTTguEBgXA=="}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:43.560413 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-46153d49-5345-4a96-bca4-0478f5e139c5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Getting quotas for project 7ff48b26c576495f8bb8c88d20d246cc. Resources: {'key_pairs'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:56:43.564927 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-46153d49-5345-4a96-bca4-0478f5e139c5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Getting quotas for user 552c23b8456a40ec8e01297aca7965d3 and project 7ff48b26c576495f8bb8c88d20d246cc. Resources: {'key_pairs'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:56:43.588309 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-46153d49-5345-4a96-bca4-0478f5e139c5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.048464 Oct 24 08:56:43.589338 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 287/581] 10.222.0.39 () {62 vars in 1200 bytes} [Mon Oct 24 08:56:43 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 50 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 24 08:56:44.078379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-12978562-fd1d-4bb0-8ce0-00df89064795 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:44.079051 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-12978562-fd1d-4bb0-8ce0-00df89064795 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:44.087030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-12978562-fd1d-4bb0-8ce0-00df89064795 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:44.087546 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-12978562-fd1d-4bb0-8ce0-00df89064795 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:44.087989 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-12978562-fd1d-4bb0-8ce0-00df89064795 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:44.088732 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-12978562-fd1d-4bb0-8ce0-00df89064795 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:44.089189 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-12978562-fd1d-4bb0-8ce0-00df89064795 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:44.089616 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-12978562-fd1d-4bb0-8ce0-00df89064795 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:44.157487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-12978562-fd1d-4bb0-8ce0-00df89064795 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:44.166879 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-12978562-fd1d-4bb0-8ce0-00df89064795 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 200 len: 1294 microversion: 2.1 time: 0.093529 Oct 24 08:56:44.167454 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 295/582] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:44 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 1294 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:44.412957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f82be3c8-ea18-4db8-ac16-c27a9872643f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:44.413898 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f82be3c8-ea18-4db8-ac16-c27a9872643f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:44.420781 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f82be3c8-ea18-4db8-ac16-c27a9872643f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:44.421372 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f82be3c8-ea18-4db8-ac16-c27a9872643f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:44.422253 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f82be3c8-ea18-4db8-ac16-c27a9872643f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:44.423309 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f82be3c8-ea18-4db8-ac16-c27a9872643f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:44.424075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f82be3c8-ea18-4db8-ac16-c27a9872643f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:44.424741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f82be3c8-ea18-4db8-ac16-c27a9872643f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:44.562234 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f82be3c8-ea18-4db8-ac16-c27a9872643f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:44.571257 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f82be3c8-ea18-4db8-ac16-c27a9872643f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1678 microversion: 2.1 time: 0.163727 Oct 24 08:56:44.571925 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 288/583] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:44 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1678 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:45.190644 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-5fc070ce-f6ff-4b01-8f76-32c4427cd569 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:45.191483 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-5fc070ce-f6ff-4b01-8f76-32c4427cd569 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:45.199000 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5fc070ce-f6ff-4b01-8f76-32c4427cd569 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:45.199553 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5fc070ce-f6ff-4b01-8f76-32c4427cd569 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:45.199983 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5fc070ce-f6ff-4b01-8f76-32c4427cd569 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:45.200893 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5fc070ce-f6ff-4b01-8f76-32c4427cd569 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:45.201357 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5fc070ce-f6ff-4b01-8f76-32c4427cd569 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:45.201960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5fc070ce-f6ff-4b01-8f76-32c4427cd569 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:45.229521 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-2097033625", "imageRef": "bba331ba-faf5-4a56-81d9-6e84e9609665", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "622e8c7b-08b8-4dcd-91fb-a2d1b63f4abe"}], "security_groups": [{"name": "tempest-securitygroup--1461178115"}], "key_name": "tempest-keypair-492103469", "user_data": "}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:45.270865 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-5fc070ce-f6ff-4b01-8f76-32c4427cd569 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:45.279769 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-5fc070ce-f6ff-4b01-8f76-32c4427cd569 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 200 len: 1300 microversion: 2.1 time: 0.092885 Oct 24 08:56:45.280669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 296/584] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:45 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 1300 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:45.368076 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.network.neutron [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] validate_networks() for [('622e8c7b-08b8-4dcd-91fb-a2d1b63f4abe', None, None, None, None, None)] {{(pid=130396) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 08:56:45.595133 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a0876d54-731e-4f32-8860-4729e5a21640 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:45.595970 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a0876d54-731e-4f32-8860-4729e5a21640 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:45.603913 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0876d54-731e-4f32-8860-4729e5a21640 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:45.604613 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0876d54-731e-4f32-8860-4729e5a21640 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:45.605238 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0876d54-731e-4f32-8860-4729e5a21640 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:45.606126 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0876d54-731e-4f32-8860-4729e5a21640 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:45.606755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0876d54-731e-4f32-8860-4729e5a21640 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:45.607358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0876d54-731e-4f32-8860-4729e5a21640 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:45.837061 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a0876d54-731e-4f32-8860-4729e5a21640 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:45.845378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a0876d54-731e-4f32-8860-4729e5a21640 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1678 microversion: 2.1 time: 0.254278 Oct 24 08:56:45.845964 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 297/585] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:45 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1678 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:45.904534 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Flavor limits 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 08:56:45.905003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Image limits 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 08:56:45.905411 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Flavor pref 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 08:56:45.905812 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Image pref 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 08:56:45.906244 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 08:56:45.918377 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:45.918983 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:45.919411 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:45.920283 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:45.920607 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:45.921028 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:45.938222 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Getting quotas for project 7ff48b26c576495f8bb8c88d20d246cc. Resources: {'ram', 'instances', 'cores'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:56:45.942680 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Getting quotas for user 552c23b8456a40ec8e01297aca7965d3 and project 7ff48b26c576495f8bb8c88d20d246cc. Resources: {'ram', 'instances', 'cores'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:56:45.952064 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=94,nova_api:DELETE=3,nova_api:UPDATE=1,nova_api:INSERT=2 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:45.954884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Going to run 1 instances... {{(pid=130396) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 08:56:45.957786 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bba331ba-faf5-4a56-81d9-6e84e9609665',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 08:56:46.014483 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-bbeb2262-55b3-43e4-9151-92aac1253879 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.791160 Oct 24 08:56:46.015316 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 289/586] 10.222.0.39 () {62 vars in 1188 bytes} [Mon Oct 24 08:56:45 2022] POST /compute/v2.1/servers => generated 402 bytes in 793 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) Oct 24 08:56:46.027009 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-24651e20-8399-426b-b3de-608cfb64c6ca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:46.027628 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-24651e20-8399-426b-b3de-608cfb64c6ca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:46.092666 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-24651e20-8399-426b-b3de-608cfb64c6ca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:46.102657 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-24651e20-8399-426b-b3de-608cfb64c6ca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:46.104633 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-24651e20-8399-426b-b3de-608cfb64c6ca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:46.106089 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-24651e20-8399-426b-b3de-608cfb64c6ca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1281 microversion: 2.20 time: 0.082802 Oct 24 08:56:46.107306 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 298/587] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:56:46 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1281 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:56:46.303090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-909d7714-8115-4339-a3b5-8a47c0818557 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:46.303793 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-909d7714-8115-4339-a3b5-8a47c0818557 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:46.310083 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-909d7714-8115-4339-a3b5-8a47c0818557 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:46.310846 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-909d7714-8115-4339-a3b5-8a47c0818557 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:46.311655 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-909d7714-8115-4339-a3b5-8a47c0818557 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:46.312198 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-909d7714-8115-4339-a3b5-8a47c0818557 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:46.312746 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-909d7714-8115-4339-a3b5-8a47c0818557 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:46.313369 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-909d7714-8115-4339-a3b5-8a47c0818557 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:46.471563 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-0ebe3ae4-c726-4979-a7de-73a1250df834 req-1809de24-8cba-4c93-a7dd-bb29e6a3c9e6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf", "tag": "9d796584-37f8-4f4f-8f8c-41d40022f76b"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:46.472867 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-909d7714-8115-4339-a3b5-8a47c0818557 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:46.482132 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-909d7714-8115-4339-a3b5-8a47c0818557 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 200 len: 1368 microversion: 2.1 time: 0.183113 Oct 24 08:56:46.489964 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-0ebe3ae4-c726-4979-a7de-73a1250df834 req-1809de24-8cba-4c93-a7dd-bb29e6a3c9e6 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:46.490863 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-0ebe3ae4-c726-4979-a7de-73a1250df834 req-1809de24-8cba-4c93-a7dd-bb29e6a3c9e6 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:46.491289 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-0ebe3ae4-c726-4979-a7de-73a1250df834 req-1809de24-8cba-4c93-a7dd-bb29e6a3c9e6 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:46.493476 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 290/588] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:46 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 1368 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:46.503823 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-0ebe3ae4-c726-4979-a7de-73a1250df834 req-1809de24-8cba-4c93-a7dd-bb29e6a3c9e6 service nova] Creating event network-changed:9d796584-37f8-4f4f-8f8c-41d40022f76b for instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf on hv-cinder-maste Oct 24 08:56:46.509936 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-0ebe3ae4-c726-4979-a7de-73a1250df834 req-1809de24-8cba-4c93-a7dd-bb29e6a3c9e6 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042241 Oct 24 08:56:46.510531 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 299/589] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:56:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:56:46.870436 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4d3ad194-72d1-46df-9449-48e7ca0b9396 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:46.871202 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4d3ad194-72d1-46df-9449-48e7ca0b9396 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:46.877763 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d3ad194-72d1-46df-9449-48e7ca0b9396 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:46.878308 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d3ad194-72d1-46df-9449-48e7ca0b9396 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:46.878826 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d3ad194-72d1-46df-9449-48e7ca0b9396 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:46.879634 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d3ad194-72d1-46df-9449-48e7ca0b9396 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:46.880172 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d3ad194-72d1-46df-9449-48e7ca0b9396 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:46.880656 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d3ad194-72d1-46df-9449-48e7ca0b9396 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:47.029180 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4d3ad194-72d1-46df-9449-48e7ca0b9396 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:47.037645 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4d3ad194-72d1-46df-9449-48e7ca0b9396 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1678 microversion: 2.1 time: 0.171527 Oct 24 08:56:47.038210 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 291/590] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:46 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1678 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:47.131668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-619aba8c-1be8-4ab9-b63f-35ad47437c82 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:47.132405 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-619aba8c-1be8-4ab9-b63f-35ad47437c82 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:47.138604 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-619aba8c-1be8-4ab9-b63f-35ad47437c82 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:47.139086 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-619aba8c-1be8-4ab9-b63f-35ad47437c82 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:47.139524 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-619aba8c-1be8-4ab9-b63f-35ad47437c82 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:47.140250 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-619aba8c-1be8-4ab9-b63f-35ad47437c82 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:47.140705 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-619aba8c-1be8-4ab9-b63f-35ad47437c82 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:47.141125 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-619aba8c-1be8-4ab9-b63f-35ad47437c82 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:47.207721 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-619aba8c-1be8-4ab9-b63f-35ad47437c82 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:47.216496 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-619aba8c-1be8-4ab9-b63f-35ad47437c82 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:47.218053 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-619aba8c-1be8-4ab9-b63f-35ad47437c82 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:47.219223 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-619aba8c-1be8-4ab9-b63f-35ad47437c82 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1333 microversion: 2.20 time: 0.092134 Oct 24 08:56:47.219787 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 300/591] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:56:47 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1333 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:56:47.509541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-3b3aec18-07f5-4a8f-a0df-95258d92a23c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:47.510055 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-3b3aec18-07f5-4a8f-a0df-95258d92a23c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:47.516903 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3b3aec18-07f5-4a8f-a0df-95258d92a23c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:47.517383 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3b3aec18-07f5-4a8f-a0df-95258d92a23c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:47.517815 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3b3aec18-07f5-4a8f-a0df-95258d92a23c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:47.518505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3b3aec18-07f5-4a8f-a0df-95258d92a23c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:47.518955 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3b3aec18-07f5-4a8f-a0df-95258d92a23c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:47.519369 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3b3aec18-07f5-4a8f-a0df-95258d92a23c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:47.657679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-3b3aec18-07f5-4a8f-a0df-95258d92a23c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:47.665796 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-3b3aec18-07f5-4a8f-a0df-95258d92a23c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 200 len: 1368 microversion: 2.1 time: 0.160738 Oct 24 08:56:47.666362 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 292/592] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:47 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 1368 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:48.060911 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-10004a90-5680-438d-9037-35c502b4735e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:48.061614 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-10004a90-5680-438d-9037-35c502b4735e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:48.067997 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-10004a90-5680-438d-9037-35c502b4735e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:48.068511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-10004a90-5680-438d-9037-35c502b4735e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:48.068952 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-10004a90-5680-438d-9037-35c502b4735e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:48.069646 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-10004a90-5680-438d-9037-35c502b4735e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:48.070099 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-10004a90-5680-438d-9037-35c502b4735e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:48.070523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-10004a90-5680-438d-9037-35c502b4735e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:48.202388 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-10004a90-5680-438d-9037-35c502b4735e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:48.210755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-10004a90-5680-438d-9037-35c502b4735e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1678 microversion: 2.1 time: 0.153913 Oct 24 08:56:48.211346 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 301/593] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:48 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1678 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:48.245654 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-97a1bb38-0a92-49ca-9422-1b3417447dbe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:48.246177 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-97a1bb38-0a92-49ca-9422-1b3417447dbe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:48.252108 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-97a1bb38-0a92-49ca-9422-1b3417447dbe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:48.252609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-97a1bb38-0a92-49ca-9422-1b3417447dbe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:48.253038 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-97a1bb38-0a92-49ca-9422-1b3417447dbe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:48.253718 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-97a1bb38-0a92-49ca-9422-1b3417447dbe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:48.254832 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-97a1bb38-0a92-49ca-9422-1b3417447dbe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:48.255432 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-97a1bb38-0a92-49ca-9422-1b3417447dbe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:48.318565 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-97a1bb38-0a92-49ca-9422-1b3417447dbe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:48.326579 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-97a1bb38-0a92-49ca-9422-1b3417447dbe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:48.328009 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-97a1bb38-0a92-49ca-9422-1b3417447dbe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:48.329115 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-97a1bb38-0a92-49ca-9422-1b3417447dbe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1341 microversion: 2.20 time: 0.087443 Oct 24 08:56:48.329664 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 293/594] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:56:48 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1341 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:56:48.689018 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c7ebba5b-d86b-47dc-8aa9-099870eeea62 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:48.689654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c7ebba5b-d86b-47dc-8aa9-099870eeea62 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:48.696631 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7ebba5b-d86b-47dc-8aa9-099870eeea62 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:48.697109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7ebba5b-d86b-47dc-8aa9-099870eeea62 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:48.697545 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7ebba5b-d86b-47dc-8aa9-099870eeea62 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:48.698234 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7ebba5b-d86b-47dc-8aa9-099870eeea62 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:48.698684 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7ebba5b-d86b-47dc-8aa9-099870eeea62 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:48.699101 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c7ebba5b-d86b-47dc-8aa9-099870eeea62 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:48.831692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c7ebba5b-d86b-47dc-8aa9-099870eeea62 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:48.840080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c7ebba5b-d86b-47dc-8aa9-099870eeea62 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 200 len: 1368 microversion: 2.1 time: 0.154843 Oct 24 08:56:48.840829 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 302/595] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:48 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 1368 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:49.235785 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b06a8a6d-d0b8-475f-ac97-c95f9fa0600a tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:49.236421 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b06a8a6d-d0b8-475f-ac97-c95f9fa0600a tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:49.243830 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b06a8a6d-d0b8-475f-ac97-c95f9fa0600a tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:49.244362 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b06a8a6d-d0b8-475f-ac97-c95f9fa0600a tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:49.244800 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b06a8a6d-d0b8-475f-ac97-c95f9fa0600a tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:49.245505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b06a8a6d-d0b8-475f-ac97-c95f9fa0600a tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:49.245961 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b06a8a6d-d0b8-475f-ac97-c95f9fa0600a tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:49.246514 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b06a8a6d-d0b8-475f-ac97-c95f9fa0600a tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:49.355587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-cc8b1726-1924-44da-8988-e6ca7e9c5c0b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:49.356240 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-cc8b1726-1924-44da-8988-e6ca7e9c5c0b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:49.363103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc8b1726-1924-44da-8988-e6ca7e9c5c0b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:49.363648 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc8b1726-1924-44da-8988-e6ca7e9c5c0b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:49.364372 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc8b1726-1924-44da-8988-e6ca7e9c5c0b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:49.365138 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc8b1726-1924-44da-8988-e6ca7e9c5c0b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:49.365632 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc8b1726-1924-44da-8988-e6ca7e9c5c0b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:49.366092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc8b1726-1924-44da-8988-e6ca7e9c5c0b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:49.384431 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b06a8a6d-d0b8-475f-ac97-c95f9fa0600a tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:49.392628 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b06a8a6d-d0b8-475f-ac97-c95f9fa0600a tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1678 microversion: 2.1 time: 0.160793 Oct 24 08:56:49.393157 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 294/596] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:49 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1678 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:49.432836 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cc8b1726-1924-44da-8988-e6ca7e9c5c0b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:49.440894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cc8b1726-1924-44da-8988-e6ca7e9c5c0b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:49.442951 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cc8b1726-1924-44da-8988-e6ca7e9c5c0b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:49.444047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-cc8b1726-1924-44da-8988-e6ca7e9c5c0b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1341 microversion: 2.20 time: 0.092367 Oct 24 08:56:49.444630 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 303/597] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:56:49 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1341 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:56:49.863631 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-935a9771-b26c-4b29-b828-5858058815f4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:49.864779 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-935a9771-b26c-4b29-b828-5858058815f4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:49.872370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-935a9771-b26c-4b29-b828-5858058815f4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:49.873237 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-935a9771-b26c-4b29-b828-5858058815f4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:49.874256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-935a9771-b26c-4b29-b828-5858058815f4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:49.875193 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-935a9771-b26c-4b29-b828-5858058815f4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:49.875999 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-935a9771-b26c-4b29-b828-5858058815f4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:49.877429 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-935a9771-b26c-4b29-b828-5858058815f4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:50.108103 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-935a9771-b26c-4b29-b828-5858058815f4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:50.117685 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-935a9771-b26c-4b29-b828-5858058815f4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 200 len: 1368 microversion: 2.1 time: 0.258380 Oct 24 08:56:50.118265 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 295/598] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:49 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 1368 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:50.417467 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-da1d9ccb-4245-4b6d-85c8-0e54beee3185 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:50.418101 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-da1d9ccb-4245-4b6d-85c8-0e54beee3185 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:50.425071 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-da1d9ccb-4245-4b6d-85c8-0e54beee3185 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:50.425629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-da1d9ccb-4245-4b6d-85c8-0e54beee3185 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:50.427050 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-da1d9ccb-4245-4b6d-85c8-0e54beee3185 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:50.427507 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-da1d9ccb-4245-4b6d-85c8-0e54beee3185 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:50.427927 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-da1d9ccb-4245-4b6d-85c8-0e54beee3185 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:50.428452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-da1d9ccb-4245-4b6d-85c8-0e54beee3185 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:50.469830 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-78b94e63-b388-4148-a8c7-bdcc8dee17c7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:50.470443 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-78b94e63-b388-4148-a8c7-bdcc8dee17c7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:50.477440 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-78b94e63-b388-4148-a8c7-bdcc8dee17c7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:50.477928 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-78b94e63-b388-4148-a8c7-bdcc8dee17c7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:50.478367 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-78b94e63-b388-4148-a8c7-bdcc8dee17c7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:50.479082 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-78b94e63-b388-4148-a8c7-bdcc8dee17c7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:50.480093 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-78b94e63-b388-4148-a8c7-bdcc8dee17c7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:50.480093 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-78b94e63-b388-4148-a8c7-bdcc8dee17c7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:50.714493 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-da1d9ccb-4245-4b6d-85c8-0e54beee3185 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:50.724240 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-da1d9ccb-4245-4b6d-85c8-0e54beee3185 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1678 microversion: 2.1 time: 0.310545 Oct 24 08:56:50.724973 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 304/599] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:50 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1678 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:50.740462 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-78b94e63-b388-4148-a8c7-bdcc8dee17c7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:50.750066 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-78b94e63-b388-4148-a8c7-bdcc8dee17c7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:50.751562 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-78b94e63-b388-4148-a8c7-bdcc8dee17c7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:50.752726 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-78b94e63-b388-4148-a8c7-bdcc8dee17c7 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1407 microversion: 2.20 time: 0.286831 Oct 24 08:56:50.753283 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 296/600] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:56:50 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1407 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:56:51.151480 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-037bb756-cbba-4c6a-af9c-4ef0dec46321 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:51.152985 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-037bb756-cbba-4c6a-af9c-4ef0dec46321 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:51.161610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-037bb756-cbba-4c6a-af9c-4ef0dec46321 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:51.162499 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-037bb756-cbba-4c6a-af9c-4ef0dec46321 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:51.163404 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-037bb756-cbba-4c6a-af9c-4ef0dec46321 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:51.164316 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-037bb756-cbba-4c6a-af9c-4ef0dec46321 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:51.164927 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-037bb756-cbba-4c6a-af9c-4ef0dec46321 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:51.165541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-037bb756-cbba-4c6a-af9c-4ef0dec46321 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:51.224940 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-be7558f9-ad41-4313-af51-d277e11289d0 req-b59f311d-6e06-48ab-9b6b-108d4738423b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf", "name": "network-vif-plugged", "status": "completed", "tag": "9d796584-37f8-4f4f-8f8c-41d40022f76b"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:51.233606 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=2,nova_api:INSERT=3 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:51.236677 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-be7558f9-ad41-4313-af51-d277e11289d0 req-b59f311d-6e06-48ab-9b6b-108d4738423b service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:51.237242 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-be7558f9-ad41-4313-af51-d277e11289d0 req-b59f311d-6e06-48ab-9b6b-108d4738423b service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:51.237778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-be7558f9-ad41-4313-af51-d277e11289d0 req-b59f311d-6e06-48ab-9b6b-108d4738423b service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:51.253092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-be7558f9-ad41-4313-af51-d277e11289d0 req-b59f311d-6e06-48ab-9b6b-108d4738423b service nova] Creating event network-vif-plugged:9d796584-37f8-4f4f-8f8c-41d40022f76b for instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf on hv-cinder-maste Oct 24 08:56:51.259293 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-be7558f9-ad41-4313-af51-d277e11289d0 req-b59f311d-6e06-48ab-9b6b-108d4738423b service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.151440 Oct 24 08:56:51.260209 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 305/601] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:56:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:56:51.359330 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-037bb756-cbba-4c6a-af9c-4ef0dec46321 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:51.369407 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-037bb756-cbba-4c6a-af9c-4ef0dec46321 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 200 len: 1368 microversion: 2.1 time: 0.225487 Oct 24 08:56:51.370750 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 297/602] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:51 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 1368 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:51.514146 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-895ed351-e773-4207-a00a-e02fd1037847 req-7c208c6f-3031-4dad-9b8d-566ae79e475a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e72acc7a-decb-46c7-9c51-f731e2c413b3", "tag": "d0e79b5c-b764-405d-8c7a-d5032e3c72a8"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:51.524676 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-895ed351-e773-4207-a00a-e02fd1037847 req-7c208c6f-3031-4dad-9b8d-566ae79e475a service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:51.525152 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-895ed351-e773-4207-a00a-e02fd1037847 req-7c208c6f-3031-4dad-9b8d-566ae79e475a service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:51.525584 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-895ed351-e773-4207-a00a-e02fd1037847 req-7c208c6f-3031-4dad-9b8d-566ae79e475a service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:51.537306 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-895ed351-e773-4207-a00a-e02fd1037847 req-7c208c6f-3031-4dad-9b8d-566ae79e475a service nova] Creating event network-changed:d0e79b5c-b764-405d-8c7a-d5032e3c72a8 for instance e72acc7a-decb-46c7-9c51-f731e2c413b3 on hv-cinder-maste Oct 24 08:56:51.542416 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-895ed351-e773-4207-a00a-e02fd1037847 req-7c208c6f-3031-4dad-9b8d-566ae79e475a service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.099219 Oct 24 08:56:51.543008 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 306/603] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:56:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:56:51.749122 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ff2ad2da-e3ff-49c5-b56f-f8681eb484ad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:51.749919 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ff2ad2da-e3ff-49c5-b56f-f8681eb484ad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:51.757869 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff2ad2da-e3ff-49c5-b56f-f8681eb484ad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:51.758500 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff2ad2da-e3ff-49c5-b56f-f8681eb484ad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:51.758953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff2ad2da-e3ff-49c5-b56f-f8681eb484ad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:51.759679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff2ad2da-e3ff-49c5-b56f-f8681eb484ad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:51.760172 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff2ad2da-e3ff-49c5-b56f-f8681eb484ad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:51.760599 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff2ad2da-e3ff-49c5-b56f-f8681eb484ad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:51.782214 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-de904bb2-869c-4053-abc1-174ff8d943de tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:51.784455 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-de904bb2-869c-4053-abc1-174ff8d943de tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:51.796602 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-de904bb2-869c-4053-abc1-174ff8d943de tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:51.797099 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-de904bb2-869c-4053-abc1-174ff8d943de tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:51.797540 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-de904bb2-869c-4053-abc1-174ff8d943de tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:51.798245 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-de904bb2-869c-4053-abc1-174ff8d943de tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:51.798703 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-de904bb2-869c-4053-abc1-174ff8d943de tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:51.799272 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-de904bb2-869c-4053-abc1-174ff8d943de tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:52.049417 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ff2ad2da-e3ff-49c5-b56f-f8681eb484ad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:52.057391 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ff2ad2da-e3ff-49c5-b56f-f8681eb484ad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1678 microversion: 2.1 time: 0.312129 Oct 24 08:56:52.057965 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 298/604] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:51 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1678 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:52.060969 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-de904bb2-869c-4053-abc1-174ff8d943de tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:52.069881 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-de904bb2-869c-4053-abc1-174ff8d943de tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:52.071386 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-de904bb2-869c-4053-abc1-174ff8d943de tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:52.072608 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-de904bb2-869c-4053-abc1-174ff8d943de tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1407 microversion: 2.20 time: 0.298553 Oct 24 08:56:52.073251 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 307/605] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:56:51 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1407 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:56:52.394193 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8fdf876b-3698-490b-9465-16c1bd482b02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:52.394822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8fdf876b-3698-490b-9465-16c1bd482b02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:52.400979 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8fdf876b-3698-490b-9465-16c1bd482b02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:52.401465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8fdf876b-3698-490b-9465-16c1bd482b02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:52.401902 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8fdf876b-3698-490b-9465-16c1bd482b02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:52.402605 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8fdf876b-3698-490b-9465-16c1bd482b02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:52.403058 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8fdf876b-3698-490b-9465-16c1bd482b02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:52.403487 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8fdf876b-3698-490b-9465-16c1bd482b02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:52.539457 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8fdf876b-3698-490b-9465-16c1bd482b02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:52.554107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8fdf876b-3698-490b-9465-16c1bd482b02 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 200 len: 1368 microversion: 2.1 time: 0.163635 Oct 24 08:56:52.554691 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 299/606] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:52 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 1368 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:53.082520 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7ee9ed04-8812-4993-9bfd-bca6931f69ac tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:53.083708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7ee9ed04-8812-4993-9bfd-bca6931f69ac tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:53.085754 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=2 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:53.091250 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7ee9ed04-8812-4993-9bfd-bca6931f69ac tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:53.091849 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7ee9ed04-8812-4993-9bfd-bca6931f69ac tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:53.092292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7ee9ed04-8812-4993-9bfd-bca6931f69ac tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:53.093044 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7ee9ed04-8812-4993-9bfd-bca6931f69ac tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:53.093504 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7ee9ed04-8812-4993-9bfd-bca6931f69ac tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:53.093935 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7ee9ed04-8812-4993-9bfd-bca6931f69ac tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:53.098840 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=100 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:53.099390 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0a88ac99-de39-47ba-9e7e-216c37895538 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:53.100094 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-0a88ac99-de39-47ba-9e7e-216c37895538 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:53.106811 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a88ac99-de39-47ba-9e7e-216c37895538 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:53.107308 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a88ac99-de39-47ba-9e7e-216c37895538 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:53.107750 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a88ac99-de39-47ba-9e7e-216c37895538 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:53.108486 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a88ac99-de39-47ba-9e7e-216c37895538 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:53.108951 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a88ac99-de39-47ba-9e7e-216c37895538 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:53.109381 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a88ac99-de39-47ba-9e7e-216c37895538 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:53.290290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7ee9ed04-8812-4993-9bfd-bca6931f69ac tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:53.298060 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7ee9ed04-8812-4993-9bfd-bca6931f69ac tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1678 microversion: 2.1 time: 0.220188 Oct 24 08:56:53.298635 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 308/607] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:53 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1678 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:53.304769 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0a88ac99-de39-47ba-9e7e-216c37895538 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:53.312818 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0a88ac99-de39-47ba-9e7e-216c37895538 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:53.314330 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0a88ac99-de39-47ba-9e7e-216c37895538 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:53.315424 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-0a88ac99-de39-47ba-9e7e-216c37895538 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1407 microversion: 2.20 time: 0.220891 Oct 24 08:56:53.315964 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 300/608] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:56:53 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1407 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:56:53.578000 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-07c7a084-63bf-4c30-b94f-e2a850e8eaa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:53.578524 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-07c7a084-63bf-4c30-b94f-e2a850e8eaa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:53.585288 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-07c7a084-63bf-4c30-b94f-e2a850e8eaa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:53.585777 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-07c7a084-63bf-4c30-b94f-e2a850e8eaa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:53.586217 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-07c7a084-63bf-4c30-b94f-e2a850e8eaa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:53.586917 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-07c7a084-63bf-4c30-b94f-e2a850e8eaa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:53.587376 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-07c7a084-63bf-4c30-b94f-e2a850e8eaa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:53.587804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-07c7a084-63bf-4c30-b94f-e2a850e8eaa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:53.719333 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-07c7a084-63bf-4c30-b94f-e2a850e8eaa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:53.727791 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-07c7a084-63bf-4c30-b94f-e2a850e8eaa3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 200 len: 1368 microversion: 2.1 time: 0.153735 Oct 24 08:56:53.728404 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 309/609] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:53 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 1368 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:54.322971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-3ef90e63-3b96-4118-b3a9-e96b3c883725 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:54.323508 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-3ef90e63-3b96-4118-b3a9-e96b3c883725 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:54.329830 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3ef90e63-3b96-4118-b3a9-e96b3c883725 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:54.330308 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3ef90e63-3b96-4118-b3a9-e96b3c883725 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:54.330736 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3ef90e63-3b96-4118-b3a9-e96b3c883725 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:54.331415 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3ef90e63-3b96-4118-b3a9-e96b3c883725 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:54.331885 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3ef90e63-3b96-4118-b3a9-e96b3c883725 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:54.333421 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3ef90e63-3b96-4118-b3a9-e96b3c883725 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:54.341202 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-cb4bf24e-da12-4367-8e7e-b1c7549f062e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:54.341819 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-cb4bf24e-da12-4367-8e7e-b1c7549f062e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:54.348332 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cb4bf24e-da12-4367-8e7e-b1c7549f062e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:54.348976 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cb4bf24e-da12-4367-8e7e-b1c7549f062e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:54.349419 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cb4bf24e-da12-4367-8e7e-b1c7549f062e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:54.350130 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cb4bf24e-da12-4367-8e7e-b1c7549f062e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:54.350589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cb4bf24e-da12-4367-8e7e-b1c7549f062e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:54.351014 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cb4bf24e-da12-4367-8e7e-b1c7549f062e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:54.352375 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:54.532857 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-3ef90e63-3b96-4118-b3a9-e96b3c883725 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:54.543251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-3ef90e63-3b96-4118-b3a9-e96b3c883725 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1678 microversion: 2.1 time: 0.224096 Oct 24 08:56:54.543927 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 301/610] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:54 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1678 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:54.544534 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cb4bf24e-da12-4367-8e7e-b1c7549f062e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:54.553016 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cb4bf24e-da12-4367-8e7e-b1c7549f062e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:54.554577 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cb4bf24e-da12-4367-8e7e-b1c7549f062e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:54.555736 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-cb4bf24e-da12-4367-8e7e-b1c7549f062e tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1407 microversion: 2.20 time: 0.218446 Oct 24 08:56:54.556316 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 310/611] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:56:54 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1407 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:56:54.751207 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ce69a499-a0bc-477e-8e2e-cfe3744a4e9a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:54.751950 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ce69a499-a0bc-477e-8e2e-cfe3744a4e9a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:54.759745 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ce69a499-a0bc-477e-8e2e-cfe3744a4e9a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:54.760268 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ce69a499-a0bc-477e-8e2e-cfe3744a4e9a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:54.760706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ce69a499-a0bc-477e-8e2e-cfe3744a4e9a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:54.761410 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ce69a499-a0bc-477e-8e2e-cfe3744a4e9a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:54.761862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ce69a499-a0bc-477e-8e2e-cfe3744a4e9a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:54.762284 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ce69a499-a0bc-477e-8e2e-cfe3744a4e9a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:54.893334 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ce69a499-a0bc-477e-8e2e-cfe3744a4e9a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:54.901353 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ce69a499-a0bc-477e-8e2e-cfe3744a4e9a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 200 len: 1368 microversion: 2.1 time: 0.154261 Oct 24 08:56:54.901912 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 302/612] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:54 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 1368 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:55.566352 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-daa21a85-5de4-40a7-92eb-f950136c803c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:55.566961 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-daa21a85-5de4-40a7-92eb-f950136c803c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:55.573508 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-daa21a85-5de4-40a7-92eb-f950136c803c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:55.574104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-daa21a85-5de4-40a7-92eb-f950136c803c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:55.574424 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-daa21a85-5de4-40a7-92eb-f950136c803c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:55.575141 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-daa21a85-5de4-40a7-92eb-f950136c803c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:55.575655 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-daa21a85-5de4-40a7-92eb-f950136c803c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:55.576102 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-daa21a85-5de4-40a7-92eb-f950136c803c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:55.581137 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-aa4af847-f396-4a47-b1eb-49589f4f996d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:55.582345 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-aa4af847-f396-4a47-b1eb-49589f4f996d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:55.588851 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aa4af847-f396-4a47-b1eb-49589f4f996d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:55.589412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aa4af847-f396-4a47-b1eb-49589f4f996d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:55.589917 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aa4af847-f396-4a47-b1eb-49589f4f996d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:55.590686 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aa4af847-f396-4a47-b1eb-49589f4f996d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:55.591144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aa4af847-f396-4a47-b1eb-49589f4f996d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:55.591570 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aa4af847-f396-4a47-b1eb-49589f4f996d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:55.792471 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-aa4af847-f396-4a47-b1eb-49589f4f996d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:55.795771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-daa21a85-5de4-40a7-92eb-f950136c803c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:55.809229 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-daa21a85-5de4-40a7-92eb-f950136c803c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1678 microversion: 2.1 time: 0.246651 Oct 24 08:56:55.810629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 311/613] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:55 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1678 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:55.812681 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-aa4af847-f396-4a47-b1eb-49589f4f996d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:55.815043 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-aa4af847-f396-4a47-b1eb-49589f4f996d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:55.817026 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-aa4af847-f396-4a47-b1eb-49589f4f996d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1407 microversion: 2.20 time: 0.239146 Oct 24 08:56:55.817534 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 303/614] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:56:55 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1407 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:56:55.924468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-6806eb5a-7a26-4292-bd19-25c1cc8038cc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:55.925071 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-6806eb5a-7a26-4292-bd19-25c1cc8038cc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:55.931492 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6806eb5a-7a26-4292-bd19-25c1cc8038cc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:55.931970 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6806eb5a-7a26-4292-bd19-25c1cc8038cc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:55.932451 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6806eb5a-7a26-4292-bd19-25c1cc8038cc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:55.933134 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6806eb5a-7a26-4292-bd19-25c1cc8038cc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:55.933604 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6806eb5a-7a26-4292-bd19-25c1cc8038cc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:55.934023 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6806eb5a-7a26-4292-bd19-25c1cc8038cc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:56.065736 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-6806eb5a-7a26-4292-bd19-25c1cc8038cc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:56.074188 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-6806eb5a-7a26-4292-bd19-25c1cc8038cc tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 200 len: 1558 microversion: 2.1 time: 0.153481 Oct 24 08:56:56.074691 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 312/615] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:55 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 1558 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:56.097312 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8b451f10-52a8-4d59-a3ed-5122b6dfd6b4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:56.097845 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8b451f10-52a8-4d59-a3ed-5122b6dfd6b4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:56.099668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=2 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:56.100607 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=16 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:56.107488 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8b451f10-52a8-4d59-a3ed-5122b6dfd6b4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:56.107978 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8b451f10-52a8-4d59-a3ed-5122b6dfd6b4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:56.108575 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8b451f10-52a8-4d59-a3ed-5122b6dfd6b4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:56.109313 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8b451f10-52a8-4d59-a3ed-5122b6dfd6b4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:56.109766 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8b451f10-52a8-4d59-a3ed-5122b6dfd6b4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:56.110187 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8b451f10-52a8-4d59-a3ed-5122b6dfd6b4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:56.193691 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8b451f10-52a8-4d59-a3ed-5122b6dfd6b4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.100228 Oct 24 08:56:56.194300 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 304/616] 10.222.0.39 () {58 vars in 1275 bytes} [Mon Oct 24 08:56:56 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf/os-interface => generated 284 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:56:56.834707 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-08b942d3-125c-4611-8df7-259f39f5be9e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:56.835344 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-08b942d3-125c-4611-8df7-259f39f5be9e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:56.837204 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=8 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:56.843325 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-08b942d3-125c-4611-8df7-259f39f5be9e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:56.843813 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-08b942d3-125c-4611-8df7-259f39f5be9e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:56.844065 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ec2930c6-4d78-4d02-b479-898d6d172bbc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:56.844738 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ec2930c6-4d78-4d02-b479-898d6d172bbc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:56.845104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-08b942d3-125c-4611-8df7-259f39f5be9e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:56.846086 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-08b942d3-125c-4611-8df7-259f39f5be9e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:56.846543 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-08b942d3-125c-4611-8df7-259f39f5be9e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:56.848063 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-08b942d3-125c-4611-8df7-259f39f5be9e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:56.852340 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ec2930c6-4d78-4d02-b479-898d6d172bbc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:56.852826 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ec2930c6-4d78-4d02-b479-898d6d172bbc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:56.853465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ec2930c6-4d78-4d02-b479-898d6d172bbc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:56.854387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ec2930c6-4d78-4d02-b479-898d6d172bbc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:56.854845 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ec2930c6-4d78-4d02-b479-898d6d172bbc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:56.855878 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ec2930c6-4d78-4d02-b479-898d6d172bbc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:57.280952 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ec2930c6-4d78-4d02-b479-898d6d172bbc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:57.288846 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-08b942d3-125c-4611-8df7-259f39f5be9e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:57.290084 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ec2930c6-4d78-4d02-b479-898d6d172bbc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:57.291695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ec2930c6-4d78-4d02-b479-898d6d172bbc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:57.293044 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ec2930c6-4d78-4d02-b479-898d6d172bbc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1407 microversion: 2.20 time: 0.453609 Oct 24 08:56:57.293640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 305/617] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:56:56 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1407 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:56:57.297656 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-08b942d3-125c-4611-8df7-259f39f5be9e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1655 microversion: 2.1 time: 0.467529 Oct 24 08:56:57.298928 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 313/618] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:56 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1655 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:57.299379 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-2e77fe40-3710-4cb3-a3ed-4640831684c2 req-44ac5774-f3cf-43a8-bda4-d6fe6ea7654f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e72acc7a-decb-46c7-9c51-f731e2c413b3", "name": "network-vif-plugged", "status": "completed", "tag": "d0e79b5c-b764-405d-8c7a-d5032e3c72a8"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:57.310737 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-2e77fe40-3710-4cb3-a3ed-4640831684c2 req-44ac5774-f3cf-43a8-bda4-d6fe6ea7654f service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:57.311949 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-2e77fe40-3710-4cb3-a3ed-4640831684c2 req-44ac5774-f3cf-43a8-bda4-d6fe6ea7654f service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:57.313034 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-2e77fe40-3710-4cb3-a3ed-4640831684c2 req-44ac5774-f3cf-43a8-bda4-d6fe6ea7654f service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:57.321303 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f3038446-9bb3-4bb0-a139-1e75fe07e4fa tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:57.328446 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f3038446-9bb3-4bb0-a139-1e75fe07e4fa tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:57.329167 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-2e77fe40-3710-4cb3-a3ed-4640831684c2 req-44ac5774-f3cf-43a8-bda4-d6fe6ea7654f service nova] Creating event network-vif-plugged:d0e79b5c-b764-405d-8c7a-d5032e3c72a8 for instance e72acc7a-decb-46c7-9c51-f731e2c413b3 on hv-cinder-maste Oct 24 08:56:57.329312 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f3038446-9bb3-4bb0-a139-1e75fe07e4fa tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:57.329599 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f3038446-9bb3-4bb0-a139-1e75fe07e4fa tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:57.334578 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-2e77fe40-3710-4cb3-a3ed-4640831684c2 req-44ac5774-f3cf-43a8-bda4-d6fe6ea7654f service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039547 Oct 24 08:56:57.335378 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 306/619] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:56:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:56:57.353107 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f3038446-9bb3-4bb0-a139-1e75fe07e4fa tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Going to try to start instance {{(pid=130395) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 24 08:56:57.398127 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f3038446-9bb3-4bb0-a139-1e75fe07e4fa tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "POST /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/action" status: 202 len: 0 microversion: 2.1 time: 0.080440 Oct 24 08:56:57.399051 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 314/620] 10.222.0.39 () {60 vars in 1278 bytes} [Mon Oct 24 08:56:57 2022] POST /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/action => generated 0 bytes in 82 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 24 08:56:57.408908 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-43ee0c13-1599-4ba0-a91d-5760c9462931 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:57.410515 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-43ee0c13-1599-4ba0-a91d-5760c9462931 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:57.417712 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-43ee0c13-1599-4ba0-a91d-5760c9462931 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:57.418598 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-43ee0c13-1599-4ba0-a91d-5760c9462931 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:57.419246 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-43ee0c13-1599-4ba0-a91d-5760c9462931 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:57.420202 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-43ee0c13-1599-4ba0-a91d-5760c9462931 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:57.420668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-43ee0c13-1599-4ba0-a91d-5760c9462931 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:57.421099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-43ee0c13-1599-4ba0-a91d-5760c9462931 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:57.742804 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-43ee0c13-1599-4ba0-a91d-5760c9462931 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:57.751281 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-43ee0c13-1599-4ba0-a91d-5760c9462931 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1664 microversion: 2.1 time: 0.346653 Oct 24 08:56:57.752457 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 307/621] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:57 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1664 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:56:57.867195 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-06d90c5b-0908-46d1-a45b-ff3c668a03b3 req-2f06ab7d-9e1c-4c47-8d3b-ad68cc8669e1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf", "tag": "9d796584-37f8-4f4f-8f8c-41d40022f76b"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:56:57.877960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-06d90c5b-0908-46d1-a45b-ff3c668a03b3 req-2f06ab7d-9e1c-4c47-8d3b-ad68cc8669e1 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:57.878747 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-06d90c5b-0908-46d1-a45b-ff3c668a03b3 req-2f06ab7d-9e1c-4c47-8d3b-ad68cc8669e1 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:57.879413 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-06d90c5b-0908-46d1-a45b-ff3c668a03b3 req-2f06ab7d-9e1c-4c47-8d3b-ad68cc8669e1 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:57.891174 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-06d90c5b-0908-46d1-a45b-ff3c668a03b3 req-2f06ab7d-9e1c-4c47-8d3b-ad68cc8669e1 service nova] Creating event network-changed:9d796584-37f8-4f4f-8f8c-41d40022f76b for instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf on hv-cinder-maste Oct 24 08:56:57.897311 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-06d90c5b-0908-46d1-a45b-ff3c668a03b3 req-2f06ab7d-9e1c-4c47-8d3b-ad68cc8669e1 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035315 Oct 24 08:56:57.897899 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 315/622] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:56:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:56:58.328318 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8b560679-1624-4fe7-bb89-f2c2d9c33bdd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:58.328928 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8b560679-1624-4fe7-bb89-f2c2d9c33bdd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:58.336102 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8b560679-1624-4fe7-bb89-f2c2d9c33bdd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:58.336727 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8b560679-1624-4fe7-bb89-f2c2d9c33bdd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:58.337169 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8b560679-1624-4fe7-bb89-f2c2d9c33bdd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:58.337915 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8b560679-1624-4fe7-bb89-f2c2d9c33bdd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:58.338403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8b560679-1624-4fe7-bb89-f2c2d9c33bdd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:58.338865 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8b560679-1624-4fe7-bb89-f2c2d9c33bdd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:58.479105 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8b560679-1624-4fe7-bb89-f2c2d9c33bdd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:58.487689 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8b560679-1624-4fe7-bb89-f2c2d9c33bdd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:58.489431 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8b560679-1624-4fe7-bb89-f2c2d9c33bdd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:58.490577 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8b560679-1624-4fe7-bb89-f2c2d9c33bdd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1407 microversion: 2.20 time: 0.166521 Oct 24 08:56:58.491119 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 308/623] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:56:58 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1407 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:56:58.774431 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8893c723-533e-4d7b-82fe-bfd7fd710473 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:58.775130 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8893c723-533e-4d7b-82fe-bfd7fd710473 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:58.781445 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8893c723-533e-4d7b-82fe-bfd7fd710473 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:58.781940 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8893c723-533e-4d7b-82fe-bfd7fd710473 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:58.782380 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8893c723-533e-4d7b-82fe-bfd7fd710473 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:58.783076 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8893c723-533e-4d7b-82fe-bfd7fd710473 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:58.783526 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8893c723-533e-4d7b-82fe-bfd7fd710473 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:58.783947 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8893c723-533e-4d7b-82fe-bfd7fd710473 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:59.264503 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8893c723-533e-4d7b-82fe-bfd7fd710473 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:59.273002 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8893c723-533e-4d7b-82fe-bfd7fd710473 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1664 microversion: 2.1 time: 0.502867 Oct 24 08:56:59.273713 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 316/624] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:56:58 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1664 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:56:59.515102 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-cf12a103-42c6-49ee-9ad3-5536520df310 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:56:59.515731 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-cf12a103-42c6-49ee-9ad3-5536520df310 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:56:59.522724 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cf12a103-42c6-49ee-9ad3-5536520df310 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:59.523203 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cf12a103-42c6-49ee-9ad3-5536520df310 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:59.523631 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cf12a103-42c6-49ee-9ad3-5536520df310 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:59.524363 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cf12a103-42c6-49ee-9ad3-5536520df310 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:56:59.524804 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cf12a103-42c6-49ee-9ad3-5536520df310 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:56:59.525220 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cf12a103-42c6-49ee-9ad3-5536520df310 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:56:59.653826 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-cf12a103-42c6-49ee-9ad3-5536520df310 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:59.661960 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-cf12a103-42c6-49ee-9ad3-5536520df310 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:59.663546 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-cf12a103-42c6-49ee-9ad3-5536520df310 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:56:59.664709 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-cf12a103-42c6-49ee-9ad3-5536520df310 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1407 microversion: 2.20 time: 0.153568 Oct 24 08:56:59.665262 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 309/625] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:56:59 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1407 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:57:00.297384 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-89ea040e-46c4-44a2-aec8-338e78bc93a8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:00.298026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-89ea040e-46c4-44a2-aec8-338e78bc93a8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:00.305247 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-89ea040e-46c4-44a2-aec8-338e78bc93a8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:00.305732 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-89ea040e-46c4-44a2-aec8-338e78bc93a8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:00.306178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-89ea040e-46c4-44a2-aec8-338e78bc93a8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:00.306889 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-89ea040e-46c4-44a2-aec8-338e78bc93a8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:00.307345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-89ea040e-46c4-44a2-aec8-338e78bc93a8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:00.307776 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-89ea040e-46c4-44a2-aec8-338e78bc93a8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:00.439037 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-89ea040e-46c4-44a2-aec8-338e78bc93a8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:00.447416 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-89ea040e-46c4-44a2-aec8-338e78bc93a8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1664 microversion: 2.1 time: 0.153868 Oct 24 08:57:00.447984 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 317/626] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:00 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1664 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:00.690910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-fbb9a090-b495-4cb2-9766-c23f1291fd98 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:00.691681 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-fbb9a090-b495-4cb2-9766-c23f1291fd98 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:00.698408 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fbb9a090-b495-4cb2-9766-c23f1291fd98 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:00.698893 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fbb9a090-b495-4cb2-9766-c23f1291fd98 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:00.699332 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fbb9a090-b495-4cb2-9766-c23f1291fd98 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:00.700060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fbb9a090-b495-4cb2-9766-c23f1291fd98 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:00.700534 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fbb9a090-b495-4cb2-9766-c23f1291fd98 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:00.700956 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fbb9a090-b495-4cb2-9766-c23f1291fd98 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:00.832656 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fbb9a090-b495-4cb2-9766-c23f1291fd98 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:00.840746 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fbb9a090-b495-4cb2-9766-c23f1291fd98 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:00.842191 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fbb9a090-b495-4cb2-9766-c23f1291fd98 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:00.843350 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-fbb9a090-b495-4cb2-9766-c23f1291fd98 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1599 microversion: 2.20 time: 0.157016 Oct 24 08:57:00.843913 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 310/627] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:57:00 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1599 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:57:00.868731 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ff8fac19-cdb2-4350-adb7-623da179dcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:00.869224 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ff8fac19-cdb2-4350-adb7-623da179dcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:00.875103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ff8fac19-cdb2-4350-adb7-623da179dcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:00.875582 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ff8fac19-cdb2-4350-adb7-623da179dcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:00.876017 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ff8fac19-cdb2-4350-adb7-623da179dcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:00.876744 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ff8fac19-cdb2-4350-adb7-623da179dcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:00.877219 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ff8fac19-cdb2-4350-adb7-623da179dcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:00.877617 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ff8fac19-cdb2-4350-adb7-623da179dcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:00.962768 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ff8fac19-cdb2-4350-adb7-623da179dcb4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.097840 Oct 24 08:57:00.963360 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 318/628] 10.222.0.39 () {60 vars in 1316 bytes} [Mon Oct 24 08:57:00 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3/os-interface => generated 284 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 24 08:57:01.020962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-392c0ee8-a767-4480-8099-7041488b30ae tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:01.021469 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-392c0ee8-a767-4480-8099-7041488b30ae tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:01.027919 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-392c0ee8-a767-4480-8099-7041488b30ae tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:01.028524 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-392c0ee8-a767-4480-8099-7041488b30ae tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:01.029036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-392c0ee8-a767-4480-8099-7041488b30ae tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:01.029844 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-392c0ee8-a767-4480-8099-7041488b30ae tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:01.030314 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-392c0ee8-a767-4480-8099-7041488b30ae tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:01.030749 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-392c0ee8-a767-4480-8099-7041488b30ae tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:01.118895 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.objects.instance [None req-392c0ee8-a767-4480-8099-7041488b30ae tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lazy-loading 'services' on Instance uuid 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf {{(pid=130396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:57:01.137413 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a5746daa-d760-45d1-a5dc-06c872cffe75 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:57:01.138213 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a5746daa-d760-45d1-a5dc-06c872cffe75 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:01.144998 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5746daa-d760-45d1-a5dc-06c872cffe75 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:01.145775 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5746daa-d760-45d1-a5dc-06c872cffe75 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:01.146395 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5746daa-d760-45d1-a5dc-06c872cffe75 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:01.147267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5746daa-d760-45d1-a5dc-06c872cffe75 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:01.147871 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5746daa-d760-45d1-a5dc-06c872cffe75 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:01.148510 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5746daa-d760-45d1-a5dc-06c872cffe75 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:01.174447 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-a5746daa-d760-45d1-a5dc-06c872cffe75 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lazy-loading 'flavor' on Instance uuid 354e90c0-1ef5-43e0-8983-2067982a3f76 {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:57:01.209853 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-392c0ee8-a767-4480-8099-7041488b30ae tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "DELETE /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/os-volume_attachments/c52e4f61-dd63-48b5-a68c-0294cbee81a6" status: 202 len: 0 microversion: 2.1 time: 0.193411 Oct 24 08:57:01.214001 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 311/629] 10.222.0.39 () {58 vars in 1416 bytes} [Mon Oct 24 08:57:01 2022] DELETE /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/os-volume_attachments/c52e4f61-dd63-48b5-a68c-0294cbee81a6 => generated 0 bytes in 195 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 08:57:01.224714 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8580bfde-5882-48b1-bd13-c55e8fd7380e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:01.228743 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8580bfde-5882-48b1-bd13-c55e8fd7380e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:01.243787 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8580bfde-5882-48b1-bd13-c55e8fd7380e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:01.244538 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8580bfde-5882-48b1-bd13-c55e8fd7380e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:01.245209 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8580bfde-5882-48b1-bd13-c55e8fd7380e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:01.246314 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8580bfde-5882-48b1-bd13-c55e8fd7380e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:01.247169 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8580bfde-5882-48b1-bd13-c55e8fd7380e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:01.248471 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8580bfde-5882-48b1-bd13-c55e8fd7380e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:01.274810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a5746daa-d760-45d1-a5dc-06c872cffe75 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "POST /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76/action" status: 202 len: 0 microversion: 2.1 time: 0.141043 Oct 24 08:57:01.276126 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 319/630] 10.222.0.39 () {60 vars in 1278 bytes} [Mon Oct 24 08:57:01 2022] POST /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76/action => generated 0 bytes in 143 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 24 08:57:01.287220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-22d36c39-8b4f-402b-9abd-8b6b3a23abb8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:01.288458 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-22d36c39-8b4f-402b-9abd-8b6b3a23abb8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:01.294978 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-22d36c39-8b4f-402b-9abd-8b6b3a23abb8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:01.295775 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-22d36c39-8b4f-402b-9abd-8b6b3a23abb8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:01.296501 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-22d36c39-8b4f-402b-9abd-8b6b3a23abb8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:01.297379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-22d36c39-8b4f-402b-9abd-8b6b3a23abb8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:01.298159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-22d36c39-8b4f-402b-9abd-8b6b3a23abb8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:01.299595 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-22d36c39-8b4f-402b-9abd-8b6b3a23abb8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:01.306055 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8580bfde-5882-48b1-bd13-c55e8fd7380e tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.089920 Oct 24 08:57:01.311357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 312/631] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 08:57:01 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/os-volume_attachments => generated 197 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:57:01.458608 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-22d36c39-8b4f-402b-9abd-8b6b3a23abb8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:01.466966 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-22d36c39-8b4f-402b-9abd-8b6b3a23abb8 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1569 microversion: 2.1 time: 0.186035 Oct 24 08:57:01.467534 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 320/632] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:01 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1569 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:57:01.470685 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-29ccec1d-acdf-4ec9-ae34-3d54d2787e28 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:01.471412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-29ccec1d-acdf-4ec9-ae34-3d54d2787e28 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:01.477761 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-29ccec1d-acdf-4ec9-ae34-3d54d2787e28 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:01.478378 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-29ccec1d-acdf-4ec9-ae34-3d54d2787e28 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:01.478932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-29ccec1d-acdf-4ec9-ae34-3d54d2787e28 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:01.479738 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-29ccec1d-acdf-4ec9-ae34-3d54d2787e28 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:01.480335 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-29ccec1d-acdf-4ec9-ae34-3d54d2787e28 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:01.480887 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-29ccec1d-acdf-4ec9-ae34-3d54d2787e28 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:01.623752 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-29ccec1d-acdf-4ec9-ae34-3d54d2787e28 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:01.632434 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-29ccec1d-acdf-4ec9-ae34-3d54d2787e28 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1668 microversion: 2.1 time: 0.165745 Oct 24 08:57:01.632913 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 313/633] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:01 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1668 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:57:01.959480 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-28b61aa7-8ad2-47da-93c1-8508f83e635c req-85d8bcba-d2a2-4be5-8313-6f2fd7d871ca service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e72acc7a-decb-46c7-9c51-f731e2c413b3", "tag": "d0e79b5c-b764-405d-8c7a-d5032e3c72a8"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:57:01.969221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-28b61aa7-8ad2-47da-93c1-8508f83e635c req-85d8bcba-d2a2-4be5-8313-6f2fd7d871ca service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:01.969727 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-28b61aa7-8ad2-47da-93c1-8508f83e635c req-85d8bcba-d2a2-4be5-8313-6f2fd7d871ca service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:01.970157 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-28b61aa7-8ad2-47da-93c1-8508f83e635c req-85d8bcba-d2a2-4be5-8313-6f2fd7d871ca service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:01.987720 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-28b61aa7-8ad2-47da-93c1-8508f83e635c req-85d8bcba-d2a2-4be5-8313-6f2fd7d871ca service nova] Creating event network-changed:d0e79b5c-b764-405d-8c7a-d5032e3c72a8 for instance e72acc7a-decb-46c7-9c51-f731e2c413b3 on hv-cinder-maste Oct 24 08:57:01.993934 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-28b61aa7-8ad2-47da-93c1-8508f83e635c req-85d8bcba-d2a2-4be5-8313-6f2fd7d871ca service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038501 Oct 24 08:57:01.996981 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 321/634] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:57:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:57:02.319981 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-394577b5-94cf-47f9-ac7d-0c65da37e8c9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:02.339799 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-394577b5-94cf-47f9-ac7d-0c65da37e8c9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:02.357945 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-394577b5-94cf-47f9-ac7d-0c65da37e8c9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:02.358425 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-394577b5-94cf-47f9-ac7d-0c65da37e8c9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:02.358868 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-394577b5-94cf-47f9-ac7d-0c65da37e8c9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:02.359585 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-394577b5-94cf-47f9-ac7d-0c65da37e8c9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:02.360115 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-394577b5-94cf-47f9-ac7d-0c65da37e8c9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:02.360798 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-394577b5-94cf-47f9-ac7d-0c65da37e8c9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:02.398461 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-394577b5-94cf-47f9-ac7d-0c65da37e8c9 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.083111 Oct 24 08:57:02.399138 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 314/635] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 08:57:02 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/os-volume_attachments => generated 197 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:57:02.492826 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1ce2fdfa-4a4e-4a24-9c8e-808a0383d066 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:02.494059 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1ce2fdfa-4a4e-4a24-9c8e-808a0383d066 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:02.501039 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ce2fdfa-4a4e-4a24-9c8e-808a0383d066 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:02.501527 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ce2fdfa-4a4e-4a24-9c8e-808a0383d066 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:02.502021 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ce2fdfa-4a4e-4a24-9c8e-808a0383d066 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:02.502739 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ce2fdfa-4a4e-4a24-9c8e-808a0383d066 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:02.503189 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ce2fdfa-4a4e-4a24-9c8e-808a0383d066 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:02.503675 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ce2fdfa-4a4e-4a24-9c8e-808a0383d066 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:02.732943 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1ce2fdfa-4a4e-4a24-9c8e-808a0383d066 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:02.742809 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1ce2fdfa-4a4e-4a24-9c8e-808a0383d066 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1569 microversion: 2.1 time: 0.255404 Oct 24 08:57:02.743877 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 322/636] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:02 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1569 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:03.413214 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ed8fe46d-e91c-4e98-8b74-3449ac93284c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:03.415379 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ed8fe46d-e91c-4e98-8b74-3449ac93284c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:03.421546 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ed8fe46d-e91c-4e98-8b74-3449ac93284c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:03.422030 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ed8fe46d-e91c-4e98-8b74-3449ac93284c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:03.422522 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ed8fe46d-e91c-4e98-8b74-3449ac93284c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:03.423250 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ed8fe46d-e91c-4e98-8b74-3449ac93284c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:03.423707 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ed8fe46d-e91c-4e98-8b74-3449ac93284c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:03.424194 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ed8fe46d-e91c-4e98-8b74-3449ac93284c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:03.455334 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ed8fe46d-e91c-4e98-8b74-3449ac93284c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047817 Oct 24 08:57:03.455891 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 315/637] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 08:57:03 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 24 08:57:03.537911 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8c1610b3-9d85-443d-8ac0-182e27997dbd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:03.538454 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8c1610b3-9d85-443d-8ac0-182e27997dbd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:03.544990 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c1610b3-9d85-443d-8ac0-182e27997dbd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:03.545469 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c1610b3-9d85-443d-8ac0-182e27997dbd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:03.545960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c1610b3-9d85-443d-8ac0-182e27997dbd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:03.546697 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c1610b3-9d85-443d-8ac0-182e27997dbd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:03.547104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c1610b3-9d85-443d-8ac0-182e27997dbd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:03.547524 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c1610b3-9d85-443d-8ac0-182e27997dbd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:03.574413 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8c1610b3-9d85-443d-8ac0-182e27997dbd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Going to try to terminate instance {{(pid=130395) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 08:57:03.651034 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8c1610b3-9d85-443d-8ac0-182e27997dbd tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "DELETE /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 204 len: 0 microversion: 2.1 time: 0.117013 Oct 24 08:57:03.651617 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 323/638] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 08:57:03 2022] DELETE /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 08:57:03.660032 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-804b6fab-f7aa-4624-8c71-df85e32925fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:03.660654 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-804b6fab-f7aa-4624-8c71-df85e32925fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:03.666503 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-804b6fab-f7aa-4624-8c71-df85e32925fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:03.666979 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-804b6fab-f7aa-4624-8c71-df85e32925fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:03.667409 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-804b6fab-f7aa-4624-8c71-df85e32925fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:03.668086 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-804b6fab-f7aa-4624-8c71-df85e32925fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:03.668558 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-804b6fab-f7aa-4624-8c71-df85e32925fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:03.668974 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-804b6fab-f7aa-4624-8c71-df85e32925fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:03.767526 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-555d254e-15c6-4932-a7e0-d6ebb956188a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:03.768328 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-555d254e-15c6-4932-a7e0-d6ebb956188a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:03.774403 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-555d254e-15c6-4932-a7e0-d6ebb956188a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:03.774896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-555d254e-15c6-4932-a7e0-d6ebb956188a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:03.775337 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-555d254e-15c6-4932-a7e0-d6ebb956188a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:03.776045 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-555d254e-15c6-4932-a7e0-d6ebb956188a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:03.776527 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-555d254e-15c6-4932-a7e0-d6ebb956188a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:03.776956 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-555d254e-15c6-4932-a7e0-d6ebb956188a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:03.803136 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-804b6fab-f7aa-4624-8c71-df85e32925fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:03.812576 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-804b6fab-f7aa-4624-8c71-df85e32925fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1690 microversion: 2.1 time: 0.156329 Oct 24 08:57:03.813260 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 316/639] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:03 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1690 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:03.916078 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-555d254e-15c6-4932-a7e0-d6ebb956188a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:03.923948 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-555d254e-15c6-4932-a7e0-d6ebb956188a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1569 microversion: 2.1 time: 0.160843 Oct 24 08:57:03.924548 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 324/640] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:03 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1569 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:57:04.837660 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7558e40a-cab4-45d5-a140-b24236295764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:04.838301 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7558e40a-cab4-45d5-a140-b24236295764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:04.844831 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7558e40a-cab4-45d5-a140-b24236295764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:04.845319 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7558e40a-cab4-45d5-a140-b24236295764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:04.845786 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7558e40a-cab4-45d5-a140-b24236295764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:04.846482 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7558e40a-cab4-45d5-a140-b24236295764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:04.846937 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7558e40a-cab4-45d5-a140-b24236295764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:04.847579 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7558e40a-cab4-45d5-a140-b24236295764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:04.947250 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e7d6cfc6-bf1c-4915-b3ae-7e3fa6f06c4d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:04.947889 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e7d6cfc6-bf1c-4915-b3ae-7e3fa6f06c4d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:04.953954 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7d6cfc6-bf1c-4915-b3ae-7e3fa6f06c4d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:04.954436 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7d6cfc6-bf1c-4915-b3ae-7e3fa6f06c4d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:04.954876 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7d6cfc6-bf1c-4915-b3ae-7e3fa6f06c4d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:04.955641 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7d6cfc6-bf1c-4915-b3ae-7e3fa6f06c4d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:04.956096 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7d6cfc6-bf1c-4915-b3ae-7e3fa6f06c4d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:04.957521 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7d6cfc6-bf1c-4915-b3ae-7e3fa6f06c4d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:04.981941 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7558e40a-cab4-45d5-a140-b24236295764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:04.990490 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7558e40a-cab4-45d5-a140-b24236295764 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1690 microversion: 2.1 time: 0.156699 Oct 24 08:57:05.001034 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 317/641] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:04 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1690 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:57:05.092330 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e7d6cfc6-bf1c-4915-b3ae-7e3fa6f06c4d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:05.100530 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e7d6cfc6-bf1c-4915-b3ae-7e3fa6f06c4d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1569 microversion: 2.1 time: 0.157117 Oct 24 08:57:05.101100 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 325/642] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:04 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1569 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:57:06.014244 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9b82b9f4-d02b-48ae-8e67-891777cb9fc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:06.014895 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9b82b9f4-d02b-48ae-8e67-891777cb9fc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:06.021981 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9b82b9f4-d02b-48ae-8e67-891777cb9fc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:06.022464 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9b82b9f4-d02b-48ae-8e67-891777cb9fc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:06.022906 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9b82b9f4-d02b-48ae-8e67-891777cb9fc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:06.023599 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9b82b9f4-d02b-48ae-8e67-891777cb9fc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:06.024053 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9b82b9f4-d02b-48ae-8e67-891777cb9fc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:06.024503 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9b82b9f4-d02b-48ae-8e67-891777cb9fc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:06.042907 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:06.124493 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a58559ec-8dbc-4992-9bea-8578773c822d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:06.125107 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a58559ec-8dbc-4992-9bea-8578773c822d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:06.131904 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a58559ec-8dbc-4992-9bea-8578773c822d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:06.132418 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a58559ec-8dbc-4992-9bea-8578773c822d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:06.132857 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a58559ec-8dbc-4992-9bea-8578773c822d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:06.133551 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a58559ec-8dbc-4992-9bea-8578773c822d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:06.134004 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a58559ec-8dbc-4992-9bea-8578773c822d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:06.134422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a58559ec-8dbc-4992-9bea-8578773c822d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:06.164622 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9b82b9f4-d02b-48ae-8e67-891777cb9fc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:06.172599 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9b82b9f4-d02b-48ae-8e67-891777cb9fc6 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1690 microversion: 2.1 time: 0.162181 Oct 24 08:57:06.173265 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 318/643] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:06 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1690 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:06.270149 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a58559ec-8dbc-4992-9bea-8578773c822d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:06.277958 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a58559ec-8dbc-4992-9bea-8578773c822d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1569 microversion: 2.1 time: 0.157307 Oct 24 08:57:06.278533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 326/644] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:06 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1569 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:57:07.197600 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8de505ef-0749-4006-a6d4-874db21cc5fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:07.198115 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8de505ef-0749-4006-a6d4-874db21cc5fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:07.204707 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8de505ef-0749-4006-a6d4-874db21cc5fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:07.205194 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8de505ef-0749-4006-a6d4-874db21cc5fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:07.205634 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8de505ef-0749-4006-a6d4-874db21cc5fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:07.206319 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8de505ef-0749-4006-a6d4-874db21cc5fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:07.207732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8de505ef-0749-4006-a6d4-874db21cc5fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:07.208230 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8de505ef-0749-4006-a6d4-874db21cc5fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:07.302558 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-389c998d-fa3f-4d1a-b237-3e67f7a35744 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:07.303208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-389c998d-fa3f-4d1a-b237-3e67f7a35744 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:07.310279 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-389c998d-fa3f-4d1a-b237-3e67f7a35744 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:07.310836 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-389c998d-fa3f-4d1a-b237-3e67f7a35744 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:07.311316 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-389c998d-fa3f-4d1a-b237-3e67f7a35744 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:07.312123 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-389c998d-fa3f-4d1a-b237-3e67f7a35744 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:07.312666 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-389c998d-fa3f-4d1a-b237-3e67f7a35744 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:07.313126 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-389c998d-fa3f-4d1a-b237-3e67f7a35744 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:07.341828 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8de505ef-0749-4006-a6d4-874db21cc5fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:07.350479 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8de505ef-0749-4006-a6d4-874db21cc5fc tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1690 microversion: 2.1 time: 0.156789 Oct 24 08:57:07.351061 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 319/645] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:07 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1690 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:07.449814 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-389c998d-fa3f-4d1a-b237-3e67f7a35744 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:07.457611 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-389c998d-fa3f-4d1a-b237-3e67f7a35744 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1569 microversion: 2.1 time: 0.159259 Oct 24 08:57:07.458181 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 327/646] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:07 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1569 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:57:08.374349 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9fad4ebd-92f4-41c7-8f2d-168b3fa65e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:08.375047 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9fad4ebd-92f4-41c7-8f2d-168b3fa65e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:08.382309 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9fad4ebd-92f4-41c7-8f2d-168b3fa65e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:08.382799 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9fad4ebd-92f4-41c7-8f2d-168b3fa65e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:08.383408 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9fad4ebd-92f4-41c7-8f2d-168b3fa65e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:08.384403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9fad4ebd-92f4-41c7-8f2d-168b3fa65e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:08.385145 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9fad4ebd-92f4-41c7-8f2d-168b3fa65e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:08.385576 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9fad4ebd-92f4-41c7-8f2d-168b3fa65e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:08.474728 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9fad4ebd-92f4-41c7-8f2d-168b3fa65e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:08.481516 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ed276d77-4b2c-4c2a-8c35-b201e017c057 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:08.483063 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ed276d77-4b2c-4c2a-8c35-b201e017c057 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:08.485362 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9fad4ebd-92f4-41c7-8f2d-168b3fa65e27 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 200 len: 1623 microversion: 2.1 time: 0.114201 Oct 24 08:57:08.486252 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 320/647] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:08 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 1623 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:08.492444 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ed276d77-4b2c-4c2a-8c35-b201e017c057 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:08.493202 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ed276d77-4b2c-4c2a-8c35-b201e017c057 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:08.493858 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ed276d77-4b2c-4c2a-8c35-b201e017c057 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:08.498172 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ed276d77-4b2c-4c2a-8c35-b201e017c057 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:08.498947 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ed276d77-4b2c-4c2a-8c35-b201e017c057 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:08.499601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ed276d77-4b2c-4c2a-8c35-b201e017c057 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:08.512636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-2a2151a0-dbbe-443e-a629-abf2e11a0559 req-a8c7b524-2fab-4a87-a88c-c2e5f46fdeee service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf", "name": "network-vif-deleted", "tag": "2a188a19-a722-46ea-96ea-89232fd07436"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:57:08.532687 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-2a2151a0-dbbe-443e-a629-abf2e11a0559 req-a8c7b524-2fab-4a87-a88c-c2e5f46fdeee service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:08.533073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-2a2151a0-dbbe-443e-a629-abf2e11a0559 req-a8c7b524-2fab-4a87-a88c-c2e5f46fdeee service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:08.533521 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-2a2151a0-dbbe-443e-a629-abf2e11a0559 req-a8c7b524-2fab-4a87-a88c-c2e5f46fdeee service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:08.559069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-2a2151a0-dbbe-443e-a629-abf2e11a0559 req-a8c7b524-2fab-4a87-a88c-c2e5f46fdeee service nova] Creating event network-vif-deleted:2a188a19-a722-46ea-96ea-89232fd07436 for instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf on hv-cinder-maste Oct 24 08:57:08.568499 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-2a2151a0-dbbe-443e-a629-abf2e11a0559 req-a8c7b524-2fab-4a87-a88c-c2e5f46fdeee service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.080793 Oct 24 08:57:08.569475 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 321/648] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:57:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:57:08.682854 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ed276d77-4b2c-4c2a-8c35-b201e017c057 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:08.693461 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ed276d77-4b2c-4c2a-8c35-b201e017c057 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1569 microversion: 2.1 time: 0.215986 Oct 24 08:57:08.694615 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 328/649] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:08 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1569 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:57:09.510851 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2473c8a8-471d-4a8c-b9c3-01e9d671380f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:09.511561 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2473c8a8-471d-4a8c-b9c3-01e9d671380f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:09.518271 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2473c8a8-471d-4a8c-b9c3-01e9d671380f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:09.518764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2473c8a8-471d-4a8c-b9c3-01e9d671380f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:09.519948 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2473c8a8-471d-4a8c-b9c3-01e9d671380f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:09.520696 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2473c8a8-471d-4a8c-b9c3-01e9d671380f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:09.521155 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2473c8a8-471d-4a8c-b9c3-01e9d671380f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:09.521588 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2473c8a8-471d-4a8c-b9c3-01e9d671380f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:09.537126 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-2473c8a8-471d-4a8c-b9c3-01e9d671380f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] HTTP exception thrown: Instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf could not be found. Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2473c8a8-471d-4a8c-b9c3-01e9d671380f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Request method failure captured: Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf HTTP/1.1 Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlHvWm_27e2NE7FPo6k_ap-1fpqaBaXju_q7yx4ngEwubmyCghP7hRGkUjD3Kh0InrG1jri_Jf3QmFeB_Ar9fw71J3GQum71pn1j3lDf69J9HnlAUxEpDFaBqRZj8Qx4DvfjLqnUdxgNhetuy9MPxaavyu2rUfHlTx9TxAVKDfIab50 Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: fd4b77e78f9d4286aa92f6f90ff8b797 Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-583791902 Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 08:57:09.538975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 08:57:09.540651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/fd4b77e78f9d4286aa92f6f90ff8b797"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_fd4b77e78f9d4286aa92f6f90ff8b797", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/fd4b77e78f9d4286aa92f6f90ff8b797"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/fd4b77e78f9d4286aa92f6f90ff8b797"}]}] Oct 24 08:57:09.540651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-ServerRescueNegativeTestJSON-583791902 Oct 24 08:57:09.540651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: fd4b77e78f9d4286aa92f6f90ff8b797 Oct 24 08:57:09.540651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-583791902 Oct 24 08:57:09.540651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-ServerRescueNegativeTestJSON-583791902-project Oct 24 08:57:09.540651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 08:57:09.540651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 08:57:09.540651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 7127613bde784330bec364f02180291e Oct 24 08:57:09.540651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-ServerRescueNegativeTestJSON-583791902-project Oct 24 08:57:09.540651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 08:57:09.540651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf could not be found. Oct 24 08:57:09.540651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': '6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf'} Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf could not be found. Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:57:09.545073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf could not be found. Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf could not be found. Oct 24 08:57:09.546822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:57:09.548181 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2473c8a8-471d-4a8c-b9c3-01e9d671380f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Returning 404 to user: Instance 6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:57:09.548181 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2473c8a8-471d-4a8c-b9c3-01e9d671380f tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf" status: 404 len: 111 microversion: 2.1 time: 0.033767 Oct 24 08:57:09.548181 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 322/650] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:09 2022] GET /compute/v2.1/servers/6cdd18d6-8c5b-46cc-bb4b-d997473c1bdf => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 08:57:09.549495 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d5794d4e-5342-4f58-b225-034e98105d5d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '.version_select at 0x7fdfedf45280>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:09.553958 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR oslo.messaging._drivers.impl_rabbit [None req-d5794d4e-5342-4f58-b225-034e98105d5d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [a2de9158-e504-4150-abf1-7641a34077df] AMQP server on 10.222.0.39:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 24 08:57:09.720255 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-54f81bcb-2412-4e12-af04-048e90febc82 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:09.720874 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-54f81bcb-2412-4e12-af04-048e90febc82 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:09.727498 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-54f81bcb-2412-4e12-af04-048e90febc82 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:09.728001 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-54f81bcb-2412-4e12-af04-048e90febc82 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:09.728450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-54f81bcb-2412-4e12-af04-048e90febc82 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:09.729139 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-54f81bcb-2412-4e12-af04-048e90febc82 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:09.729585 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-54f81bcb-2412-4e12-af04-048e90febc82 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:09.730004 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-54f81bcb-2412-4e12-af04-048e90febc82 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:09.884339 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-54f81bcb-2412-4e12-af04-048e90febc82 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:09.892013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-54f81bcb-2412-4e12-af04-048e90febc82 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1569 microversion: 2.1 time: 0.175995 Oct 24 08:57:09.892609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 323/651] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:09 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1569 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:57:10.561994 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO oslo.messaging._drivers.impl_rabbit [None req-d5794d4e-5342-4f58-b225-034e98105d5d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [a2de9158-e504-4150-abf1-7641a34077df] Reconnected to AMQP server on 10.222.0.39:5672 via [amqp] client with port 34386. Oct 24 08:57:10.582821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d5794d4e-5342-4f58-b225-034e98105d5d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1226907291" status: 202 len: 0 microversion: 2.1 time: 1.036887 Oct 24 08:57:10.583410 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 329/652] 10.222.0.39 () {58 vars in 1221 bytes} [Mon Oct 24 08:57:09 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1226907291 => generated 0 bytes in 1038 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 08:57:10.916031 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-63d38d70-480d-41e6-a0e7-dd57db6511d9 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:10.916663 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-63d38d70-480d-41e6-a0e7-dd57db6511d9 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:10.924435 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-63d38d70-480d-41e6-a0e7-dd57db6511d9 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:10.924928 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-63d38d70-480d-41e6-a0e7-dd57db6511d9 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:10.926109 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-63d38d70-480d-41e6-a0e7-dd57db6511d9 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:10.926821 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-63d38d70-480d-41e6-a0e7-dd57db6511d9 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:10.927279 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-63d38d70-480d-41e6-a0e7-dd57db6511d9 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:10.927704 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-63d38d70-480d-41e6-a0e7-dd57db6511d9 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:11.137744 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-63d38d70-480d-41e6-a0e7-dd57db6511d9 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:11.152727 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-63d38d70-480d-41e6-a0e7-dd57db6511d9 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1569 microversion: 2.1 time: 0.240446 Oct 24 08:57:11.153744 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 324/653] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:10 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1569 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:12.178119 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d2ee56fa-beaf-4d4f-bec6-11f401c34c52 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:12.178801 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d2ee56fa-beaf-4d4f-bec6-11f401c34c52 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:12.186209 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d2ee56fa-beaf-4d4f-bec6-11f401c34c52 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:12.186694 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d2ee56fa-beaf-4d4f-bec6-11f401c34c52 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:12.187138 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d2ee56fa-beaf-4d4f-bec6-11f401c34c52 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:12.187841 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d2ee56fa-beaf-4d4f-bec6-11f401c34c52 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:12.188341 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d2ee56fa-beaf-4d4f-bec6-11f401c34c52 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:12.188772 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d2ee56fa-beaf-4d4f-bec6-11f401c34c52 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:12.322229 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d2ee56fa-beaf-4d4f-bec6-11f401c34c52 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:12.330183 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d2ee56fa-beaf-4d4f-bec6-11f401c34c52 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1569 microversion: 2.1 time: 0.156840 Oct 24 08:57:12.331062 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 330/654] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:12 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1569 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:13.355444 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5375c558-8ec1-4ce1-9de0-2514f062d70b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:13.356064 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-5375c558-8ec1-4ce1-9de0-2514f062d70b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:13.362543 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5375c558-8ec1-4ce1-9de0-2514f062d70b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:13.363177 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5375c558-8ec1-4ce1-9de0-2514f062d70b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:13.363447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5375c558-8ec1-4ce1-9de0-2514f062d70b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:13.364438 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5375c558-8ec1-4ce1-9de0-2514f062d70b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:13.364893 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5375c558-8ec1-4ce1-9de0-2514f062d70b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:13.365317 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5375c558-8ec1-4ce1-9de0-2514f062d70b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:13.499601 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5375c558-8ec1-4ce1-9de0-2514f062d70b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:13.507427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5375c558-8ec1-4ce1-9de0-2514f062d70b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1569 microversion: 2.1 time: 0.155981 Oct 24 08:57:13.508019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 325/655] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:13 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1569 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:13.571843 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-2f278bf6-e8b9-46d1-b920-2ef0d7f1c695 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:13.572359 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-2f278bf6-e8b9-46d1-b920-2ef0d7f1c695 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:13.578724 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2f278bf6-e8b9-46d1-b920-2ef0d7f1c695 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:13.579204 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2f278bf6-e8b9-46d1-b920-2ef0d7f1c695 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:13.579633 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2f278bf6-e8b9-46d1-b920-2ef0d7f1c695 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:13.580346 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2f278bf6-e8b9-46d1-b920-2ef0d7f1c695 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:13.580788 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2f278bf6-e8b9-46d1-b920-2ef0d7f1c695 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:13.581737 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2f278bf6-e8b9-46d1-b920-2ef0d7f1c695 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:13.607807 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-2f278bf6-e8b9-46d1-b920-2ef0d7f1c695 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Going to try to terminate instance {{(pid=130395) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 08:57:13.685650 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-2f278bf6-e8b9-46d1-b920-2ef0d7f1c695 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "DELETE /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 204 len: 0 microversion: 2.1 time: 0.117276 Oct 24 08:57:13.686230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 331/656] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 08:57:13 2022] DELETE /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 08:57:13.694486 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-76988210-7239-4a90-ba40-1feba9d0f1b2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:13.695129 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-76988210-7239-4a90-ba40-1feba9d0f1b2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:13.701261 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76988210-7239-4a90-ba40-1feba9d0f1b2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:13.701769 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76988210-7239-4a90-ba40-1feba9d0f1b2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:13.702192 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76988210-7239-4a90-ba40-1feba9d0f1b2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:13.702891 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76988210-7239-4a90-ba40-1feba9d0f1b2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:13.703346 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76988210-7239-4a90-ba40-1feba9d0f1b2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:13.703775 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-76988210-7239-4a90-ba40-1feba9d0f1b2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:13.837689 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-76988210-7239-4a90-ba40-1feba9d0f1b2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:13.845313 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-76988210-7239-4a90-ba40-1feba9d0f1b2 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1508 microversion: 2.1 time: 0.154872 Oct 24 08:57:13.845870 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 326/657] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:13 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1508 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:14.531447 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-3639d852-7c14-4d74-9bae-195c204f246a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:14.532178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-3639d852-7c14-4d74-9bae-195c204f246a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:14.538090 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3639d852-7c14-4d74-9bae-195c204f246a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:14.538572 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3639d852-7c14-4d74-9bae-195c204f246a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:14.539016 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3639d852-7c14-4d74-9bae-195c204f246a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:14.539704 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3639d852-7c14-4d74-9bae-195c204f246a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:14.540182 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3639d852-7c14-4d74-9bae-195c204f246a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:14.540614 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3639d852-7c14-4d74-9bae-195c204f246a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:14.671016 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-3639d852-7c14-4d74-9bae-195c204f246a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:14.679077 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-3639d852-7c14-4d74-9bae-195c204f246a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1569 microversion: 2.1 time: 0.151909 Oct 24 08:57:14.679670 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 332/658] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:14 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1569 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:14.869913 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-47a13fff-58c0-407f-ac0b-0153538b8496 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:14.870528 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-47a13fff-58c0-407f-ac0b-0153538b8496 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:14.876984 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47a13fff-58c0-407f-ac0b-0153538b8496 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:14.877471 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47a13fff-58c0-407f-ac0b-0153538b8496 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:14.877911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47a13fff-58c0-407f-ac0b-0153538b8496 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:14.879305 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47a13fff-58c0-407f-ac0b-0153538b8496 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:14.879759 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47a13fff-58c0-407f-ac0b-0153538b8496 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:14.880224 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-47a13fff-58c0-407f-ac0b-0153538b8496 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:15.263238 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-47a13fff-58c0-407f-ac0b-0153538b8496 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:15.271635 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-47a13fff-58c0-407f-ac0b-0153538b8496 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1508 microversion: 2.1 time: 0.405613 Oct 24 08:57:15.272270 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 327/659] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:14 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1508 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:15.703647 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-3bde44c9-4f35-4442-ac4a-fddb5648d369 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:15.704616 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-3bde44c9-4f35-4442-ac4a-fddb5648d369 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:15.712897 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3bde44c9-4f35-4442-ac4a-fddb5648d369 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:15.713396 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3bde44c9-4f35-4442-ac4a-fddb5648d369 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:15.713842 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3bde44c9-4f35-4442-ac4a-fddb5648d369 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:15.714596 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3bde44c9-4f35-4442-ac4a-fddb5648d369 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:15.715466 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3bde44c9-4f35-4442-ac4a-fddb5648d369 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:15.715989 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3bde44c9-4f35-4442-ac4a-fddb5648d369 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:15.943703 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-3bde44c9-4f35-4442-ac4a-fddb5648d369 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:15.951764 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-3bde44c9-4f35-4442-ac4a-fddb5648d369 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1569 microversion: 2.1 time: 0.253000 Oct 24 08:57:15.952518 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 333/660] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:15 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1569 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:16.258491 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-40020941-b9c8-41bc-9be4-010f7eb3c749 req-54b37596-7f16-4189-a023-32f6de5546db service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0ec5fcd8-ee4b-4143-90be-991e1da8936e", "name": "network-vif-deleted", "tag": "3a7e5724-5c3f-4f69-8404-1022cccfec37"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:57:16.264008 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=78,nova_api:INSERT=3 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:16.271607 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-40020941-b9c8-41bc-9be4-010f7eb3c749 req-54b37596-7f16-4189-a023-32f6de5546db service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:16.272085 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-40020941-b9c8-41bc-9be4-010f7eb3c749 req-54b37596-7f16-4189-a023-32f6de5546db service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:16.272555 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-40020941-b9c8-41bc-9be4-010f7eb3c749 req-54b37596-7f16-4189-a023-32f6de5546db service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:16.283638 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-40020941-b9c8-41bc-9be4-010f7eb3c749 req-54b37596-7f16-4189-a023-32f6de5546db service nova] Creating event network-vif-deleted:3a7e5724-5c3f-4f69-8404-1022cccfec37 for instance 0ec5fcd8-ee4b-4143-90be-991e1da8936e on hv-cinder-maste Oct 24 08:57:16.289077 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-40020941-b9c8-41bc-9be4-010f7eb3c749 req-54b37596-7f16-4189-a023-32f6de5546db service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034229 Oct 24 08:57:16.289846 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 328/661] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:57:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:57:16.295975 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a682ecdb-f6d5-4fb9-b119-a2262fb78558 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:16.296637 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a682ecdb-f6d5-4fb9-b119-a2262fb78558 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:16.302567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a682ecdb-f6d5-4fb9-b119-a2262fb78558 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:16.303058 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a682ecdb-f6d5-4fb9-b119-a2262fb78558 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:16.303497 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a682ecdb-f6d5-4fb9-b119-a2262fb78558 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:16.304253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a682ecdb-f6d5-4fb9-b119-a2262fb78558 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:16.304707 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a682ecdb-f6d5-4fb9-b119-a2262fb78558 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:16.305138 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a682ecdb-f6d5-4fb9-b119-a2262fb78558 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:16.372410 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a682ecdb-f6d5-4fb9-b119-a2262fb78558 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:16.382157 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a682ecdb-f6d5-4fb9-b119-a2262fb78558 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 200 len: 1466 microversion: 2.1 time: 0.089924 Oct 24 08:57:16.382742 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 334/662] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:16 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 1466 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:16.979833 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-edcacea8-2354-44ae-a7e3-6057fac08ba3 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:16.980670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-edcacea8-2354-44ae-a7e3-6057fac08ba3 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:16.988256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-edcacea8-2354-44ae-a7e3-6057fac08ba3 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:16.988256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-edcacea8-2354-44ae-a7e3-6057fac08ba3 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:16.988695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-edcacea8-2354-44ae-a7e3-6057fac08ba3 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:16.989673 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-edcacea8-2354-44ae-a7e3-6057fac08ba3 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:16.990123 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-edcacea8-2354-44ae-a7e3-6057fac08ba3 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:16.990545 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-edcacea8-2354-44ae-a7e3-6057fac08ba3 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:17.135967 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-edcacea8-2354-44ae-a7e3-6057fac08ba3 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:17.144609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-edcacea8-2354-44ae-a7e3-6057fac08ba3 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1510 microversion: 2.1 time: 0.171809 Oct 24 08:57:17.145197 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 329/663] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:16 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1510 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:17.170223 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-33fd36c2-21e3-4926-a069-57c32241369a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:57:17.170851 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-33fd36c2-21e3-4926-a069-57c32241369a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:17.177509 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-33fd36c2-21e3-4926-a069-57c32241369a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:17.177998 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-33fd36c2-21e3-4926-a069-57c32241369a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:17.178439 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-33fd36c2-21e3-4926-a069-57c32241369a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:17.179144 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-33fd36c2-21e3-4926-a069-57c32241369a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:17.179602 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-33fd36c2-21e3-4926-a069-57c32241369a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:17.180038 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-33fd36c2-21e3-4926-a069-57c32241369a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:17.209386 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-33fd36c2-21e3-4926-a069-57c32241369a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=130395) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 24 08:57:17.273168 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-8066372f-31d9-4819-be63-dd3b3da1e52b req-956f3ccb-49c6-475d-a0bf-acb444ae0212 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "354e90c0-1ef5-43e0-8983-2067982a3f76", "name": "network-vif-unplugged", "status": "completed", "tag": "4a9dc16c-b094-43ae-a495-1098e5c9f054"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:57:17.284412 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-33fd36c2-21e3-4926-a069-57c32241369a tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "POST /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76/action" status: 202 len: 0 microversion: 2.1 time: 0.117850 Oct 24 08:57:17.285969 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 335/664] 10.222.0.39 () {60 vars in 1278 bytes} [Mon Oct 24 08:57:17 2022] POST /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76/action => generated 0 bytes in 120 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 24 08:57:17.293657 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b08e2aed-4126-40e2-8443-350b7dd5636e tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:17.294344 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b08e2aed-4126-40e2-8443-350b7dd5636e tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:17.294928 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-8066372f-31d9-4819-be63-dd3b3da1e52b req-956f3ccb-49c6-475d-a0bf-acb444ae0212 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:17.295424 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-8066372f-31d9-4819-be63-dd3b3da1e52b req-956f3ccb-49c6-475d-a0bf-acb444ae0212 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:17.295860 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-8066372f-31d9-4819-be63-dd3b3da1e52b req-956f3ccb-49c6-475d-a0bf-acb444ae0212 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:17.302503 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b08e2aed-4126-40e2-8443-350b7dd5636e tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:17.303004 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b08e2aed-4126-40e2-8443-350b7dd5636e tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:17.303448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b08e2aed-4126-40e2-8443-350b7dd5636e tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:17.304375 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b08e2aed-4126-40e2-8443-350b7dd5636e tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:17.304870 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b08e2aed-4126-40e2-8443-350b7dd5636e tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:17.305442 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b08e2aed-4126-40e2-8443-350b7dd5636e tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:17.308478 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.compute.server_external_events [req-8066372f-31d9-4819-be63-dd3b3da1e52b req-956f3ccb-49c6-475d-a0bf-acb444ae0212 service nova] Unable to find a host for instance 354e90c0-1ef5-43e0-8983-2067982a3f76. Dropping event network-vif-unplugged {{(pid=130396) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 24 08:57:17.310101 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-8066372f-31d9-4819-be63-dd3b3da1e52b req-956f3ccb-49c6-475d-a0bf-acb444ae0212 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.044982 Oct 24 08:57:17.311189 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 330/665] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:57:17 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 47 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Oct 24 08:57:17.406341 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-bd65e767-e67f-4d22-97a1-99a7c8c86a23 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:17.407211 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-bd65e767-e67f-4d22-97a1-99a7c8c86a23 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: 0ec5fcd8-ee4b-4143-90be-991e1da8936e] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:17.414471 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bd65e767-e67f-4d22-97a1-99a7c8c86a23 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:17.415163 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bd65e767-e67f-4d22-97a1-99a7c8c86a23 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:17.415895 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bd65e767-e67f-4d22-97a1-99a7c8c86a23 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:17.416824 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bd65e767-e67f-4d22-97a1-99a7c8c86a23 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:17.417289 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bd65e767-e67f-4d22-97a1-99a7c8c86a23 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:17.417718 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bd65e767-e67f-4d22-97a1-99a7c8c86a23 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:17.434066 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-bd65e767-e67f-4d22-97a1-99a7c8c86a23 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] HTTP exception thrown: Instance 0ec5fcd8-ee4b-4143-90be-991e1da8936e could not be found. Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-bd65e767-e67f-4d22-97a1-99a7c8c86a23 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Request method failure captured: Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e HTTP/1.1 Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlHvWm_27e2NE7FPo6k_ap-1fpqaBaXju_q7yx4ngEwubmyCghP7hRGkUjD3Kh0InrG1jri_Jf3QmFeB_Ar9fw71J3GQum71pn1j3lDf69J9HnlAUxEpDFaBqRZj8Qx4DvfjLqnUdxgNhetuy9MPxaavyu2rUfHlTx9TxAVKDfIab50 Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: fd4b77e78f9d4286aa92f6f90ff8b797 Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-583791902 Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 08:57:17.435754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 08:57:17.437690 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/fd4b77e78f9d4286aa92f6f90ff8b797"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_fd4b77e78f9d4286aa92f6f90ff8b797", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/fd4b77e78f9d4286aa92f6f90ff8b797"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/fd4b77e78f9d4286aa92f6f90ff8b797"}]}] Oct 24 08:57:17.437690 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-ServerRescueNegativeTestJSON-583791902 Oct 24 08:57:17.437690 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: fd4b77e78f9d4286aa92f6f90ff8b797 Oct 24 08:57:17.437690 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-583791902 Oct 24 08:57:17.437690 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-ServerRescueNegativeTestJSON-583791902-project Oct 24 08:57:17.437690 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 08:57:17.437690 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 08:57:17.437690 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 7127613bde784330bec364f02180291e Oct 24 08:57:17.437690 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-ServerRescueNegativeTestJSON-583791902-project Oct 24 08:57:17.437690 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 08:57:17.437690 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance 0ec5fcd8-ee4b-4143-90be-991e1da8936e could not be found. Oct 24 08:57:17.437690 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': '0ec5fcd8-ee4b-4143-90be-991e1da8936e'} Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0ec5fcd8-ee4b-4143-90be-991e1da8936e could not be found. Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:57:17.438567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0ec5fcd8-ee4b-4143-90be-991e1da8936e could not be found. Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0ec5fcd8-ee4b-4143-90be-991e1da8936e could not be found. Oct 24 08:57:17.440039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:57:17.441897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-bd65e767-e67f-4d22-97a1-99a7c8c86a23 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Returning 404 to user: Instance 0ec5fcd8-ee4b-4143-90be-991e1da8936e could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:57:17.441897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-bd65e767-e67f-4d22-97a1-99a7c8c86a23 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e" status: 404 len: 111 microversion: 2.1 time: 0.035460 Oct 24 08:57:17.441897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 331/666] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:17 2022] GET /compute/v2.1/servers/0ec5fcd8-ee4b-4143-90be-991e1da8936e => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 24 08:57:17.450834 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6473e3da-860c-4d5b-89b2-e9212ab9398d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:17.451335 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6473e3da-860c-4d5b-89b2-e9212ab9398d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:17.458426 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6473e3da-860c-4d5b-89b2-e9212ab9398d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:17.458917 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6473e3da-860c-4d5b-89b2-e9212ab9398d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:17.459352 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6473e3da-860c-4d5b-89b2-e9212ab9398d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:17.460318 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6473e3da-860c-4d5b-89b2-e9212ab9398d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:17.460771 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6473e3da-860c-4d5b-89b2-e9212ab9398d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:17.461193 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6473e3da-860c-4d5b-89b2-e9212ab9398d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:17.474381 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-b08e2aed-4126-40e2-8443-350b7dd5636e tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:17.482382 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:17.488286 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b08e2aed-4126-40e2-8443-350b7dd5636e tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1518 microversion: 2.1 time: 0.198491 Oct 24 08:57:17.488879 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 336/667] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:17 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1518 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:17.491111 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6473e3da-860c-4d5b-89b2-e9212ab9398d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Going to try to terminate instance {{(pid=130396) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 08:57:17.578204 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6473e3da-860c-4d5b-89b2-e9212ab9398d tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "DELETE /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 204 len: 0 microversion: 2.1 time: 0.130920 Oct 24 08:57:17.579147 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 332/668] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 08:57:17 2022] DELETE /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 24 08:57:17.589936 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-dbb3a8f2-8606-43e0-8431-9642be3139f5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:17.590646 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-dbb3a8f2-8606-43e0-8431-9642be3139f5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:17.598545 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dbb3a8f2-8606-43e0-8431-9642be3139f5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:17.599038 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dbb3a8f2-8606-43e0-8431-9642be3139f5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:17.599474 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dbb3a8f2-8606-43e0-8431-9642be3139f5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:17.600332 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dbb3a8f2-8606-43e0-8431-9642be3139f5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:17.600791 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dbb3a8f2-8606-43e0-8431-9642be3139f5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:17.601217 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dbb3a8f2-8606-43e0-8431-9642be3139f5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:17.748922 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-dbb3a8f2-8606-43e0-8431-9642be3139f5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:17.758189 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-dbb3a8f2-8606-43e0-8431-9642be3139f5 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.172121 Oct 24 08:57:17.758880 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 337/669] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:17 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:18.514093 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-067813d0-d461-4b6e-a6bd-a8bce779adda tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:18.514867 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-067813d0-d461-4b6e-a6bd-a8bce779adda tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:18.523358 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-067813d0-d461-4b6e-a6bd-a8bce779adda tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:18.524211 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-067813d0-d461-4b6e-a6bd-a8bce779adda tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:18.525266 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-067813d0-d461-4b6e-a6bd-a8bce779adda tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:18.526486 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-067813d0-d461-4b6e-a6bd-a8bce779adda tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:18.527030 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-067813d0-d461-4b6e-a6bd-a8bce779adda tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:18.527632 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-067813d0-d461-4b6e-a6bd-a8bce779adda tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:18.670123 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-067813d0-d461-4b6e-a6bd-a8bce779adda tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:18.679046 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-067813d0-d461-4b6e-a6bd-a8bce779adda tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1518 microversion: 2.1 time: 0.169494 Oct 24 08:57:18.679629 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 333/670] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:18 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1518 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:57:18.783298 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-cc214414-7061-41da-b63a-726d31f6e37c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:18.783818 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-cc214414-7061-41da-b63a-726d31f6e37c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:18.791121 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc214414-7061-41da-b63a-726d31f6e37c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:18.791609 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc214414-7061-41da-b63a-726d31f6e37c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:18.792054 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc214414-7061-41da-b63a-726d31f6e37c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:18.792799 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc214414-7061-41da-b63a-726d31f6e37c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:18.793258 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc214414-7061-41da-b63a-726d31f6e37c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:18.793688 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cc214414-7061-41da-b63a-726d31f6e37c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:18.927366 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cc214414-7061-41da-b63a-726d31f6e37c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:18.935305 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-cc214414-7061-41da-b63a-726d31f6e37c tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.156092 Oct 24 08:57:18.935907 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 338/671] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:18 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:19.702924 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6f635757-ccb3-4694-8c34-50577410660f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:19.703565 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6f635757-ccb3-4694-8c34-50577410660f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:19.711006 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6f635757-ccb3-4694-8c34-50577410660f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:19.711482 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6f635757-ccb3-4694-8c34-50577410660f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:19.711911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6f635757-ccb3-4694-8c34-50577410660f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:19.712622 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6f635757-ccb3-4694-8c34-50577410660f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:19.713068 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6f635757-ccb3-4694-8c34-50577410660f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:19.713485 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6f635757-ccb3-4694-8c34-50577410660f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:19.869434 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6f635757-ccb3-4694-8c34-50577410660f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:19.877656 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6f635757-ccb3-4694-8c34-50577410660f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1518 microversion: 2.1 time: 0.178485 Oct 24 08:57:19.878772 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 334/672] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:19 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1518 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:19.959095 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c2d50c35-07a9-4d55-b08e-4b6728d3feac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:19.959680 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c2d50c35-07a9-4d55-b08e-4b6728d3feac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:19.966499 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c2d50c35-07a9-4d55-b08e-4b6728d3feac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:19.967082 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c2d50c35-07a9-4d55-b08e-4b6728d3feac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:19.967411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c2d50c35-07a9-4d55-b08e-4b6728d3feac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:19.968124 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c2d50c35-07a9-4d55-b08e-4b6728d3feac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:19.968603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c2d50c35-07a9-4d55-b08e-4b6728d3feac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:19.969638 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c2d50c35-07a9-4d55-b08e-4b6728d3feac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:20.101954 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c2d50c35-07a9-4d55-b08e-4b6728d3feac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:20.109587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c2d50c35-07a9-4d55-b08e-4b6728d3feac tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1522 microversion: 2.1 time: 0.155109 Oct 24 08:57:20.110139 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 339/673] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:19 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:20.906994 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-05afea13-7dc5-4b6f-a46f-b36fcf851e9b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:20.907819 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-05afea13-7dc5-4b6f-a46f-b36fcf851e9b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:20.919671 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05afea13-7dc5-4b6f-a46f-b36fcf851e9b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:20.920296 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05afea13-7dc5-4b6f-a46f-b36fcf851e9b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:20.922923 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05afea13-7dc5-4b6f-a46f-b36fcf851e9b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:20.926979 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05afea13-7dc5-4b6f-a46f-b36fcf851e9b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:20.927884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05afea13-7dc5-4b6f-a46f-b36fcf851e9b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:20.928572 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05afea13-7dc5-4b6f-a46f-b36fcf851e9b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:20.957025 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-914f8f0a-0a38-4011-bf69-6ba37e33fadb req-e33cbc02-5bbd-4120-97b4-ac1b5cd856b1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e7b4b4a1-6d10-4eef-94b3-395c2fc1c451", "name": "network-vif-deleted", "tag": "cab5d319-dc72-4203-90f4-1a5c76d81460"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:57:20.967963 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-914f8f0a-0a38-4011-bf69-6ba37e33fadb req-e33cbc02-5bbd-4120-97b4-ac1b5cd856b1 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:20.968643 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-914f8f0a-0a38-4011-bf69-6ba37e33fadb req-e33cbc02-5bbd-4120-97b4-ac1b5cd856b1 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:20.968903 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-914f8f0a-0a38-4011-bf69-6ba37e33fadb req-e33cbc02-5bbd-4120-97b4-ac1b5cd856b1 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:20.981757 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-914f8f0a-0a38-4011-bf69-6ba37e33fadb req-e33cbc02-5bbd-4120-97b4-ac1b5cd856b1 service nova] Creating event network-vif-deleted:cab5d319-dc72-4203-90f4-1a5c76d81460 for instance e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 on hv-cinder-maste Oct 24 08:57:20.987030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-914f8f0a-0a38-4011-bf69-6ba37e33fadb req-e33cbc02-5bbd-4120-97b4-ac1b5cd856b1 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033611 Oct 24 08:57:20.988515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 340/674] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:57:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:57:21.084849 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-05afea13-7dc5-4b6f-a46f-b36fcf851e9b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:21.093248 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-05afea13-7dc5-4b6f-a46f-b36fcf851e9b tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1576 microversion: 2.1 time: 0.193285 Oct 24 08:57:21.094124 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 335/675] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:20 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1576 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:21.134562 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-831d70aa-cd15-4772-bf35-5f83d1435269 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:21.135261 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-831d70aa-cd15-4772-bf35-5f83d1435269 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:21.141783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-831d70aa-cd15-4772-bf35-5f83d1435269 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:21.142247 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-831d70aa-cd15-4772-bf35-5f83d1435269 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:21.142684 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-831d70aa-cd15-4772-bf35-5f83d1435269 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:21.143393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-831d70aa-cd15-4772-bf35-5f83d1435269 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:21.144666 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-831d70aa-cd15-4772-bf35-5f83d1435269 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:21.145110 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-831d70aa-cd15-4772-bf35-5f83d1435269 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:21.181355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-831d70aa-cd15-4772-bf35-5f83d1435269 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:21.181994 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-831d70aa-cd15-4772-bf35-5f83d1435269 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:21.182551 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-831d70aa-cd15-4772-bf35-5f83d1435269 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:21.182881 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-831d70aa-cd15-4772-bf35-5f83d1435269 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lazy-loading 'fault' on Instance uuid e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:57:21.235381 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-831d70aa-cd15-4772-bf35-5f83d1435269 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7127613bde784330bec364f02180291e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd4b77e78f9d4286aa92f6f90ff8b797', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:21.246464 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-831d70aa-cd15-4772-bf35-5f83d1435269 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 200 len: 1312 microversion: 2.1 time: 0.115968 Oct 24 08:57:21.256579 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 341/676] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:21 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 1312 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:22.118958 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-32bc51e2-d954-49c4-8f60-cb947141c839 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:22.119662 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-32bc51e2-d954-49c4-8f60-cb947141c839 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:22.127468 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32bc51e2-d954-49c4-8f60-cb947141c839 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:22.133856 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32bc51e2-d954-49c4-8f60-cb947141c839 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:22.133856 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32bc51e2-d954-49c4-8f60-cb947141c839 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:22.133856 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32bc51e2-d954-49c4-8f60-cb947141c839 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:22.133856 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32bc51e2-d954-49c4-8f60-cb947141c839 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:22.134376 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32bc51e2-d954-49c4-8f60-cb947141c839 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:22.269831 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-11a82c6c-070a-4286-8c9e-36d81840b303 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:22.270448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-11a82c6c-070a-4286-8c9e-36d81840b303 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] [instance: e7b4b4a1-6d10-4eef-94b3-395c2fc1c451] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:22.272894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=90,nova_api:UPDATE=3,nova_api:DELETE=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:22.278674 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-11a82c6c-070a-4286-8c9e-36d81840b303 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:22.279232 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-11a82c6c-070a-4286-8c9e-36d81840b303 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:22.279713 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-11a82c6c-070a-4286-8c9e-36d81840b303 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:22.280445 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-11a82c6c-070a-4286-8c9e-36d81840b303 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:22.280909 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-11a82c6c-070a-4286-8c9e-36d81840b303 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:22.281336 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-11a82c6c-070a-4286-8c9e-36d81840b303 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:22.299327 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-11a82c6c-070a-4286-8c9e-36d81840b303 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] HTTP exception thrown: Instance e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 could not be found. Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-11a82c6c-070a-4286-8c9e-36d81840b303 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Request method failure captured: Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 HTTP/1.1 Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlHvWm_27e2NE7FPo6k_ap-1fpqaBaXju_q7yx4ngEwubmyCghP7hRGkUjD3Kh0InrG1jri_Jf3QmFeB_Ar9fw71J3GQum71pn1j3lDf69J9HnlAUxEpDFaBqRZj8Qx4DvfjLqnUdxgNhetuy9MPxaavyu2rUfHlTx9TxAVKDfIab50 Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: fd4b77e78f9d4286aa92f6f90ff8b797 Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-583791902 Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 08:57:22.301755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 08:57:22.303282 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/fd4b77e78f9d4286aa92f6f90ff8b797"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_fd4b77e78f9d4286aa92f6f90ff8b797", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/fd4b77e78f9d4286aa92f6f90ff8b797"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/fd4b77e78f9d4286aa92f6f90ff8b797"}]}] Oct 24 08:57:22.303282 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-ServerRescueNegativeTestJSON-583791902 Oct 24 08:57:22.303282 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: fd4b77e78f9d4286aa92f6f90ff8b797 Oct 24 08:57:22.303282 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-583791902 Oct 24 08:57:22.303282 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-ServerRescueNegativeTestJSON-583791902-project Oct 24 08:57:22.303282 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 08:57:22.303282 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 08:57:22.303282 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: 7127613bde784330bec364f02180291e Oct 24 08:57:22.303282 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-ServerRescueNegativeTestJSON-583791902-project Oct 24 08:57:22.303282 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 08:57:22.303282 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 could not be found. Oct 24 08:57:22.303282 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': 'e7b4b4a1-6d10-4eef-94b3-395c2fc1c451'} Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 could not be found. Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:57:22.304103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 could not be found. Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 could not be found. Oct 24 08:57:22.305533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:57:22.306744 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-11a82c6c-070a-4286-8c9e-36d81840b303 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] Returning 404 to user: Instance e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:57:22.306744 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-11a82c6c-070a-4286-8c9e-36d81840b303 tempest-ServerRescueNegativeTestJSON-583791902 tempest-ServerRescueNegativeTestJSON-583791902-project] 10.222.0.39 "GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451" status: 404 len: 111 microversion: 2.1 time: 0.037526 Oct 24 08:57:22.306744 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 342/677] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:22 2022] GET /compute/v2.1/servers/e7b4b4a1-6d10-4eef-94b3-395c2fc1c451 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 24 08:57:22.378773 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-32bc51e2-d954-49c4-8f60-cb947141c839 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:22.382624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:22.388233 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-32bc51e2-d954-49c4-8f60-cb947141c839 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1576 microversion: 2.1 time: 0.273271 Oct 24 08:57:22.389333 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 336/678] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:22 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1576 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:23.413240 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c38ebeee-4d8a-4dd0-b191-676643627dea tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:23.413844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c38ebeee-4d8a-4dd0-b191-676643627dea tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:23.420518 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c38ebeee-4d8a-4dd0-b191-676643627dea tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:23.421007 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c38ebeee-4d8a-4dd0-b191-676643627dea tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:23.421449 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c38ebeee-4d8a-4dd0-b191-676643627dea tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:23.422165 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c38ebeee-4d8a-4dd0-b191-676643627dea tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:23.422623 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c38ebeee-4d8a-4dd0-b191-676643627dea tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:23.423051 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c38ebeee-4d8a-4dd0-b191-676643627dea tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:23.562197 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c38ebeee-4d8a-4dd0-b191-676643627dea tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:23.571719 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c38ebeee-4d8a-4dd0-b191-676643627dea tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1576 microversion: 2.1 time: 0.162937 Oct 24 08:57:23.572471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 343/679] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:23 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1576 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:24.600734 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-be26fe8d-474e-46dc-be3a-6e24c1bc0a2d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:24.601370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-be26fe8d-474e-46dc-be3a-6e24c1bc0a2d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:24.609821 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-be26fe8d-474e-46dc-be3a-6e24c1bc0a2d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:24.614698 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-be26fe8d-474e-46dc-be3a-6e24c1bc0a2d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:24.615463 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-be26fe8d-474e-46dc-be3a-6e24c1bc0a2d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:24.616242 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-be26fe8d-474e-46dc-be3a-6e24c1bc0a2d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:24.616704 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-be26fe8d-474e-46dc-be3a-6e24c1bc0a2d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:24.617126 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-be26fe8d-474e-46dc-be3a-6e24c1bc0a2d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:24.780824 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-be26fe8d-474e-46dc-be3a-6e24c1bc0a2d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:24.805874 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-be26fe8d-474e-46dc-be3a-6e24c1bc0a2d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1576 microversion: 2.1 time: 0.210800 Oct 24 08:57:24.806348 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 337/680] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:24 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1576 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:25.836795 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ccc3a495-c959-4068-9650-69d1580397de tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:25.837941 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ccc3a495-c959-4068-9650-69d1580397de tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:25.855081 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ccc3a495-c959-4068-9650-69d1580397de tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:25.856054 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ccc3a495-c959-4068-9650-69d1580397de tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:25.856762 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ccc3a495-c959-4068-9650-69d1580397de tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:25.857794 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ccc3a495-c959-4068-9650-69d1580397de tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:25.858621 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ccc3a495-c959-4068-9650-69d1580397de tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:25.859385 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ccc3a495-c959-4068-9650-69d1580397de tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:26.063740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ccc3a495-c959-4068-9650-69d1580397de tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:26.074557 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ccc3a495-c959-4068-9650-69d1580397de tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1576 microversion: 2.1 time: 0.247823 Oct 24 08:57:26.075298 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 344/681] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:25 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1576 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:27.101269 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-d9922616-9ee9-44b7-9285-8a5b8d4a9a6c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:27.101885 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-d9922616-9ee9-44b7-9285-8a5b8d4a9a6c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:27.108617 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d9922616-9ee9-44b7-9285-8a5b8d4a9a6c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:27.109107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d9922616-9ee9-44b7-9285-8a5b8d4a9a6c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:27.109658 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d9922616-9ee9-44b7-9285-8a5b8d4a9a6c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:27.110365 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d9922616-9ee9-44b7-9285-8a5b8d4a9a6c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:27.110826 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d9922616-9ee9-44b7-9285-8a5b8d4a9a6c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:27.111254 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d9922616-9ee9-44b7-9285-8a5b8d4a9a6c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:27.262627 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d9922616-9ee9-44b7-9285-8a5b8d4a9a6c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:27.275972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-d9922616-9ee9-44b7-9285-8a5b8d4a9a6c tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1576 microversion: 2.1 time: 0.178580 Oct 24 08:57:27.276606 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 338/682] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:27 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1576 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:28.302949 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-29ab8984-e932-4d80-959f-600edf385669 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:28.303652 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-29ab8984-e932-4d80-959f-600edf385669 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:28.310051 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-29ab8984-e932-4d80-959f-600edf385669 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:28.310535 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-29ab8984-e932-4d80-959f-600edf385669 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:28.310965 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-29ab8984-e932-4d80-959f-600edf385669 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:28.311658 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-29ab8984-e932-4d80-959f-600edf385669 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:28.312114 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-29ab8984-e932-4d80-959f-600edf385669 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:28.312558 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-29ab8984-e932-4d80-959f-600edf385669 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:28.490226 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-29ab8984-e932-4d80-959f-600edf385669 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:28.498798 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-29ab8984-e932-4d80-959f-600edf385669 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1576 microversion: 2.1 time: 0.200824 Oct 24 08:57:28.499514 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 345/683] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:28 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1576 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:29.525567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2aebe100-eed5-44b3-a4da-d15442ebad16 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:29.526091 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2aebe100-eed5-44b3-a4da-d15442ebad16 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:29.533086 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2aebe100-eed5-44b3-a4da-d15442ebad16 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:29.533595 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2aebe100-eed5-44b3-a4da-d15442ebad16 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:29.534171 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2aebe100-eed5-44b3-a4da-d15442ebad16 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:29.534841 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2aebe100-eed5-44b3-a4da-d15442ebad16 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:29.535297 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2aebe100-eed5-44b3-a4da-d15442ebad16 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:29.535723 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2aebe100-eed5-44b3-a4da-d15442ebad16 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:29.692224 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2aebe100-eed5-44b3-a4da-d15442ebad16 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:29.700287 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2aebe100-eed5-44b3-a4da-d15442ebad16 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1576 microversion: 2.1 time: 0.178648 Oct 24 08:57:29.700872 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 339/684] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:29 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1576 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:30.726235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a1435df7-200c-4858-8ed9-7a3a24f9f2ec tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:30.726836 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a1435df7-200c-4858-8ed9-7a3a24f9f2ec tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:30.733698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a1435df7-200c-4858-8ed9-7a3a24f9f2ec tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:30.734191 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a1435df7-200c-4858-8ed9-7a3a24f9f2ec tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:30.734639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a1435df7-200c-4858-8ed9-7a3a24f9f2ec tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:30.735342 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a1435df7-200c-4858-8ed9-7a3a24f9f2ec tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:30.735807 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a1435df7-200c-4858-8ed9-7a3a24f9f2ec tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:30.736264 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a1435df7-200c-4858-8ed9-7a3a24f9f2ec tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:30.877229 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a1435df7-200c-4858-8ed9-7a3a24f9f2ec tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:30.885950 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a1435df7-200c-4858-8ed9-7a3a24f9f2ec tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1576 microversion: 2.1 time: 0.163539 Oct 24 08:57:30.886541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 346/685] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:30 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1576 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:31.909126 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0bf9c30f-716b-44ae-8ad9-78b4554700e2 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:31.910088 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-0bf9c30f-716b-44ae-8ad9-78b4554700e2 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:31.917594 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0bf9c30f-716b-44ae-8ad9-78b4554700e2 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:31.918229 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0bf9c30f-716b-44ae-8ad9-78b4554700e2 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:31.919038 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0bf9c30f-716b-44ae-8ad9-78b4554700e2 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:31.920019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0bf9c30f-716b-44ae-8ad9-78b4554700e2 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:31.920574 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0bf9c30f-716b-44ae-8ad9-78b4554700e2 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:31.921229 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0bf9c30f-716b-44ae-8ad9-78b4554700e2 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:32.086849 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0bf9c30f-716b-44ae-8ad9-78b4554700e2 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:32.097114 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-0bf9c30f-716b-44ae-8ad9-78b4554700e2 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1576 microversion: 2.1 time: 0.191514 Oct 24 08:57:32.098122 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 340/686] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:31 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1576 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:33.122218 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-6188316f-338a-49ad-9dae-800c242eccbd tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:33.122831 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-6188316f-338a-49ad-9dae-800c242eccbd tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:33.133242 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6188316f-338a-49ad-9dae-800c242eccbd tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:33.134227 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6188316f-338a-49ad-9dae-800c242eccbd tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:33.134827 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6188316f-338a-49ad-9dae-800c242eccbd tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:33.135568 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6188316f-338a-49ad-9dae-800c242eccbd tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:33.136180 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6188316f-338a-49ad-9dae-800c242eccbd tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:33.136648 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6188316f-338a-49ad-9dae-800c242eccbd tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:33.317714 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-6188316f-338a-49ad-9dae-800c242eccbd tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:33.326724 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-6188316f-338a-49ad-9dae-800c242eccbd tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1576 microversion: 2.1 time: 0.208405 Oct 24 08:57:33.327308 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 347/687] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:33 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1576 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:33.398511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-c4efe804-1fec-469b-923c-88e6cfe4f50e req-cf30ab31-8f35-46e0-bad3-47d7dc6cc47a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "354e90c0-1ef5-43e0-8983-2067982a3f76", "name": "network-vif-plugged", "status": "completed", "tag": "4a9dc16c-b094-43ae-a495-1098e5c9f054"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:57:33.414024 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-c4efe804-1fec-469b-923c-88e6cfe4f50e req-cf30ab31-8f35-46e0-bad3-47d7dc6cc47a service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:33.414714 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-c4efe804-1fec-469b-923c-88e6cfe4f50e req-cf30ab31-8f35-46e0-bad3-47d7dc6cc47a service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:33.415454 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-c4efe804-1fec-469b-923c-88e6cfe4f50e req-cf30ab31-8f35-46e0-bad3-47d7dc6cc47a service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:33.430392 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-c4efe804-1fec-469b-923c-88e6cfe4f50e req-cf30ab31-8f35-46e0-bad3-47d7dc6cc47a service nova] Creating event network-vif-plugged:4a9dc16c-b094-43ae-a495-1098e5c9f054 for instance 354e90c0-1ef5-43e0-8983-2067982a3f76 on hv-cinder-maste Oct 24 08:57:33.436535 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-c4efe804-1fec-469b-923c-88e6cfe4f50e req-cf30ab31-8f35-46e0-bad3-47d7dc6cc47a service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044050 Oct 24 08:57:33.437426 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 341/688] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:57:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:57:34.357034 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e1475b5c-013e-486e-b5ae-04afd2302296 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:34.357648 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e1475b5c-013e-486e-b5ae-04afd2302296 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:34.366401 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e1475b5c-013e-486e-b5ae-04afd2302296 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:34.366977 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e1475b5c-013e-486e-b5ae-04afd2302296 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:34.367409 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e1475b5c-013e-486e-b5ae-04afd2302296 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:34.368115 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e1475b5c-013e-486e-b5ae-04afd2302296 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:34.368591 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e1475b5c-013e-486e-b5ae-04afd2302296 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:34.369012 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e1475b5c-013e-486e-b5ae-04afd2302296 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:34.537048 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e1475b5c-013e-486e-b5ae-04afd2302296 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:34.547391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e1475b5c-013e-486e-b5ae-04afd2302296 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1576 microversion: 2.1 time: 0.194509 Oct 24 08:57:34.548126 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 348/689] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:34 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1576 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:35.573302 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e8aceff4-7754-4eda-b691-352536b70bf0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:35.574001 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e8aceff4-7754-4eda-b691-352536b70bf0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:35.580506 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e8aceff4-7754-4eda-b691-352536b70bf0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:35.581909 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e8aceff4-7754-4eda-b691-352536b70bf0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:35.582511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e8aceff4-7754-4eda-b691-352536b70bf0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:35.583221 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e8aceff4-7754-4eda-b691-352536b70bf0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:35.583710 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e8aceff4-7754-4eda-b691-352536b70bf0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:35.584077 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e8aceff4-7754-4eda-b691-352536b70bf0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:35.755723 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e8aceff4-7754-4eda-b691-352536b70bf0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:35.763798 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e8aceff4-7754-4eda-b691-352536b70bf0 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1576 microversion: 2.1 time: 0.195228 Oct 24 08:57:35.764415 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 342/690] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:35 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1576 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:57:36.789834 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d7044cec-1249-4dce-948d-b83731aa24fb tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:57:36.790584 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d7044cec-1249-4dce-948d-b83731aa24fb tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:57:36.798870 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d7044cec-1249-4dce-948d-b83731aa24fb tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:36.799843 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d7044cec-1249-4dce-948d-b83731aa24fb tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:36.800678 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d7044cec-1249-4dce-948d-b83731aa24fb tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:36.801276 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d7044cec-1249-4dce-948d-b83731aa24fb tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:57:36.801734 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d7044cec-1249-4dce-948d-b83731aa24fb tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:57:36.802219 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d7044cec-1249-4dce-948d-b83731aa24fb tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:57:36.947195 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d7044cec-1249-4dce-948d-b83731aa24fb tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:57:36.956366 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d7044cec-1249-4dce-948d-b83731aa24fb tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1563 microversion: 2.1 time: 0.170383 Oct 24 08:57:36.957038 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 349/691] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:57:36 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1563 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:58:25.353948 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a7e2d271-2c07-4f31-9a00-31eb44b45394 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:25.356082 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-a7e2d271-2c07-4f31-9a00-31eb44b45394 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:25.358361 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=34,nova_api:UPDATE=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:58:25.359162 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=32 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:58:25.367841 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a7e2d271-2c07-4f31-9a00-31eb44b45394 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:25.368367 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a7e2d271-2c07-4f31-9a00-31eb44b45394 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:25.368808 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a7e2d271-2c07-4f31-9a00-31eb44b45394 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:25.369508 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a7e2d271-2c07-4f31-9a00-31eb44b45394 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:25.369964 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a7e2d271-2c07-4f31-9a00-31eb44b45394 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:25.370393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a7e2d271-2c07-4f31-9a00-31eb44b45394 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:25.401090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-a7e2d271-2c07-4f31-9a00-31eb44b45394 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.051000 Oct 24 08:58:25.401650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 343/692] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 08:58:25 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528/os-volume_attachments => generated 25 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 24 08:58:25.464302 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f4021938-5dbe-45a9-8989-6c469355b189 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:25.464828 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f4021938-5dbe-45a9-8989-6c469355b189 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:25.467117 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=15 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:58:25.467810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=72 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:58:25.475291 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f4021938-5dbe-45a9-8989-6c469355b189 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:25.475780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f4021938-5dbe-45a9-8989-6c469355b189 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:25.476252 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f4021938-5dbe-45a9-8989-6c469355b189 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:25.476961 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f4021938-5dbe-45a9-8989-6c469355b189 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:25.477418 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f4021938-5dbe-45a9-8989-6c469355b189 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:25.477854 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f4021938-5dbe-45a9-8989-6c469355b189 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:25.502949 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f4021938-5dbe-45a9-8989-6c469355b189 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Going to try to terminate instance {{(pid=130395) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 08:58:25.585684 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f4021938-5dbe-45a9-8989-6c469355b189 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "DELETE /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 204 len: 0 microversion: 2.1 time: 0.125011 Oct 24 08:58:25.590009 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 350/693] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 08:58:25 2022] DELETE /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 08:58:25.596207 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7fb42528-ffb9-4190-8518-031c6272fb30 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:25.596825 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7fb42528-ffb9-4190-8518-031c6272fb30 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:25.602757 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb42528-ffb9-4190-8518-031c6272fb30 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:25.603246 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb42528-ffb9-4190-8518-031c6272fb30 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:25.603688 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb42528-ffb9-4190-8518-031c6272fb30 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:25.604418 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb42528-ffb9-4190-8518-031c6272fb30 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:25.604874 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb42528-ffb9-4190-8518-031c6272fb30 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:25.605299 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb42528-ffb9-4190-8518-031c6272fb30 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:25.738269 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7fb42528-ffb9-4190-8518-031c6272fb30 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:25.746331 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7fb42528-ffb9-4190-8518-031c6272fb30 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1674 microversion: 2.1 time: 0.155019 Oct 24 08:58:25.746897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 344/694] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:25 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1674 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:58:26.772292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-5a8cd997-c9c2-43de-92c1-8818d22a33b8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:26.772905 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-5a8cd997-c9c2-43de-92c1-8818d22a33b8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:26.779372 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5a8cd997-c9c2-43de-92c1-8818d22a33b8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:26.779857 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5a8cd997-c9c2-43de-92c1-8818d22a33b8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:26.780342 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5a8cd997-c9c2-43de-92c1-8818d22a33b8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:26.781053 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5a8cd997-c9c2-43de-92c1-8818d22a33b8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:26.781508 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5a8cd997-c9c2-43de-92c1-8818d22a33b8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:26.782037 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5a8cd997-c9c2-43de-92c1-8818d22a33b8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:26.914881 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-5a8cd997-c9c2-43de-92c1-8818d22a33b8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:26.923271 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-5a8cd997-c9c2-43de-92c1-8818d22a33b8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1674 microversion: 2.1 time: 0.154757 Oct 24 08:58:26.923959 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 351/695] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:26 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1674 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:27.946952 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-338bcea5-6ae5-426d-92ed-b772091ad82f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:27.947483 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-338bcea5-6ae5-426d-92ed-b772091ad82f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:27.954609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-338bcea5-6ae5-426d-92ed-b772091ad82f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:27.955166 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-338bcea5-6ae5-426d-92ed-b772091ad82f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:27.955281 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-338bcea5-6ae5-426d-92ed-b772091ad82f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:27.956147 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-338bcea5-6ae5-426d-92ed-b772091ad82f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:27.956615 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-338bcea5-6ae5-426d-92ed-b772091ad82f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:27.957069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-338bcea5-6ae5-426d-92ed-b772091ad82f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:28.061835 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-338bcea5-6ae5-426d-92ed-b772091ad82f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:28.073392 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-338bcea5-6ae5-426d-92ed-b772091ad82f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1606 microversion: 2.1 time: 0.130151 Oct 24 08:58:28.089007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 345/696] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:27 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1606 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:28.376926 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-cecda1b8-f32a-4b7f-a53c-8a52288a8a7f req-ec64ae5d-b6e6-49c4-93ba-91c541a5bf57 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "89a2295a-90e9-4fc1-be3b-a130f7e71528", "name": "network-vif-deleted", "tag": "47ad5200-c751-4a26-a72e-3a49c982245c"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:58:28.388724 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-cecda1b8-f32a-4b7f-a53c-8a52288a8a7f req-ec64ae5d-b6e6-49c4-93ba-91c541a5bf57 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:28.389470 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-cecda1b8-f32a-4b7f-a53c-8a52288a8a7f req-ec64ae5d-b6e6-49c4-93ba-91c541a5bf57 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:28.390163 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-cecda1b8-f32a-4b7f-a53c-8a52288a8a7f req-ec64ae5d-b6e6-49c4-93ba-91c541a5bf57 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:28.404571 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-cecda1b8-f32a-4b7f-a53c-8a52288a8a7f req-ec64ae5d-b6e6-49c4-93ba-91c541a5bf57 service nova] Creating event network-vif-deleted:47ad5200-c751-4a26-a72e-3a49c982245c for instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 on hv-cinder-maste Oct 24 08:58:28.410765 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-cecda1b8-f32a-4b7f-a53c-8a52288a8a7f req-ec64ae5d-b6e6-49c4-93ba-91c541a5bf57 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044800 Oct 24 08:58:28.411699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 352/697] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:58:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:58:29.100172 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8a2af96d-e8ab-4d62-8dc9-423ab0a33436 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:29.100912 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8a2af96d-e8ab-4d62-8dc9-423ab0a33436 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:29.108493 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8a2af96d-e8ab-4d62-8dc9-423ab0a33436 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:29.108938 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8a2af96d-e8ab-4d62-8dc9-423ab0a33436 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:29.109338 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8a2af96d-e8ab-4d62-8dc9-423ab0a33436 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:29.110932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8a2af96d-e8ab-4d62-8dc9-423ab0a33436 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:29.111461 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8a2af96d-e8ab-4d62-8dc9-423ab0a33436 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:29.111892 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8a2af96d-e8ab-4d62-8dc9-423ab0a33436 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:29.149084 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8a2af96d-e8ab-4d62-8dc9-423ab0a33436 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:29.149569 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8a2af96d-e8ab-4d62-8dc9-423ab0a33436 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:29.150008 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8a2af96d-e8ab-4d62-8dc9-423ab0a33436 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:29.150460 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.objects.instance [None req-8a2af96d-e8ab-4d62-8dc9-423ab0a33436 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lazy-loading 'fault' on Instance uuid 89a2295a-90e9-4fc1-be3b-a130f7e71528 {{(pid=130396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:58:29.213604 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8a2af96d-e8ab-4d62-8dc9-423ab0a33436 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:29.221298 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8a2af96d-e8ab-4d62-8dc9-423ab0a33436 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 200 len: 1328 microversion: 2.1 time: 0.124854 Oct 24 08:58:29.221802 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 346/698] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:29 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 1328 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:30.248035 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-71b285e8-5f24-4912-939f-cedad4909a88 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:30.248691 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-71b285e8-5f24-4912-939f-cedad4909a88 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:30.255317 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-71b285e8-5f24-4912-939f-cedad4909a88 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:30.255794 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-71b285e8-5f24-4912-939f-cedad4909a88 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:30.256256 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-71b285e8-5f24-4912-939f-cedad4909a88 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:30.256942 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-71b285e8-5f24-4912-939f-cedad4909a88 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:30.257391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-71b285e8-5f24-4912-939f-cedad4909a88 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:30.257811 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-71b285e8-5f24-4912-939f-cedad4909a88 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:30.271813 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-71b285e8-5f24-4912-939f-cedad4909a88 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] HTTP exception thrown: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-71b285e8-5f24-4912-939f-cedad4909a88 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Request method failure captured: Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 HTTP/1.1 Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlJb9biqZLl4tWim6-VtdE2wWG9xRCxMCbYJwKuM8w9jeHPhVjvEAVBfMCX4oubDLKjVnEHUr-ThOh3GW94chJTe-e8dcvllpAzphf2yg8VKeTt2KTF7-NusT3oCU-alk_qv54dC8z12P3bjjNWfhm3BLDPH-75ZWBuVhbBIUEzWS_8 Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: 45be9d45ba4c4a2ca1ed053a7263dbee Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-AttachVolumeTestJSON-791554923 Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 08:58:30.273468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 08:58:30.275208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/45be9d45ba4c4a2ca1ed053a7263dbee"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_45be9d45ba4c4a2ca1ed053a7263dbee", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/45be9d45ba4c4a2ca1ed053a7263dbee"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/45be9d45ba4c4a2ca1ed053a7263dbee"}]}] Oct 24 08:58:30.275208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-AttachVolumeTestJSON-791554923 Oct 24 08:58:30.275208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: 45be9d45ba4c4a2ca1ed053a7263dbee Oct 24 08:58:30.275208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-AttachVolumeTestJSON-791554923 Oct 24 08:58:30.275208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-AttachVolumeTestJSON-791554923-project Oct 24 08:58:30.275208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 08:58:30.275208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 08:58:30.275208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: 94169e17266b4e6fad7e068a4c1384c7 Oct 24 08:58:30.275208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-AttachVolumeTestJSON-791554923-project Oct 24 08:58:30.275208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 08:58:30.275208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. Oct 24 08:58:30.275208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': '89a2295a-90e9-4fc1-be3b-a130f7e71528'} Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:58:30.276112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. Oct 24 08:58:30.277654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:58:30.279008 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-71b285e8-5f24-4912-939f-cedad4909a88 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Returning 404 to user: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:58:30.279008 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-71b285e8-5f24-4912-939f-cedad4909a88 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 404 len: 111 microversion: 2.1 time: 0.031580 Oct 24 08:58:30.279008 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 353/699] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:30 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 08:58:30.284994 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-bb593a6e-ee27-4914-a39b-d8bd7144984e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '.version_select at 0x7fdfedc4f790>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:30.304439 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-bb593a6e-ee27-4914-a39b-d8bd7144984e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1643738377" status: 202 len: 0 microversion: 2.1 time: 0.024340 Oct 24 08:58:30.304996 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 347/700] 10.222.0.39 () {58 vars in 1221 bytes} [Mon Oct 24 08:58:30 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1643738377 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 24 08:58:30.941139 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-55468562-62dc-44ac-bd1a-fb2ad1dd9aac tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Action: 'create', calling method: .version_select at 0x7fdfede5cc10>, body: {"keypair": {"name": "tempest-keypair-1933428633", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEIh9oI64wGQZJSflSVQ9nQVAF42+BzA8EDCJD68CUgIuI3unCR7sPR4/O3sQCT0xucJojn4pMYvZr5PBmxUyYRh8P1it5pAANqi+QNkicurgDJzFl0HJjdTqmOPh7D8ZA=="}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:58:30.956842 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-55468562-62dc-44ac-bd1a-fb2ad1dd9aac tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Getting quotas for project 45be9d45ba4c4a2ca1ed053a7263dbee. Resources: {'key_pairs'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:58:30.960730 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-55468562-62dc-44ac-bd1a-fb2ad1dd9aac tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Getting quotas for user 94169e17266b4e6fad7e068a4c1384c7 and project 45be9d45ba4c4a2ca1ed053a7263dbee. Resources: {'key_pairs'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:58:30.985768 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-55468562-62dc-44ac-bd1a-fb2ad1dd9aac tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.048581 Oct 24 08:58:30.986430 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 354/701] 10.222.0.39 () {60 vars in 1159 bytes} [Mon Oct 24 08:58:30 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:58:31.252188 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1353536826", "imageRef": "bba331ba-faf5-4a56-81d9-6e84e9609665", "flavorRef": "452", "networks": [{"uuid": "d6be167f-3870-4d45-a512-3d97a0f21398"}]}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:58:31.318095 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.network.neutron [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] validate_networks() for [('d6be167f-3870-4d45-a512-3d97a0f21398', None, None, None, None, None)] {{(pid=130396) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 08:58:31.685734 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Flavor limits 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 08:58:31.686153 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Image limits 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 08:58:31.686625 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Flavor pref 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 08:58:31.687028 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Image pref 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 08:58:31.687474 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 08:58:31.697276 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:31.697761 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:31.698230 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:31.699016 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:31.699471 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:31.699935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:31.717346 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Getting quotas for project 01318d53c6ed499e94a25e12f826719f. Resources: {'ram', 'instances', 'cores'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:58:31.721838 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Getting quotas for user c94f6e2fa2b94630a745af7c54ec680a and project 01318d53c6ed499e94a25e12f826719f. Resources: {'ram', 'instances', 'cores'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:58:31.732810 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Going to run 1 instances... {{(pid=130396) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 08:58:31.735789 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bba331ba-faf5-4a56-81d9-6e84e9609665',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 08:58:31.780827 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f9521a1d-43ab-48d0-b913-f395bb388a2d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.532579 Oct 24 08:58:31.781521 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 348/702] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 08:58:31 2022] POST /compute/v2.1/servers => generated 380 bytes in 534 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 08:58:31.793854 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7e5dd905-e87c-428c-99b2-2e437c765d0a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:31.795347 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7e5dd905-e87c-428c-99b2-2e437c765d0a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:31.866672 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7e5dd905-e87c-428c-99b2-2e437c765d0a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:31.876083 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7e5dd905-e87c-428c-99b2-2e437c765d0a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1219 microversion: 2.1 time: 0.086034 Oct 24 08:58:31.877122 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 355/703] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:31 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:32.372984 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-506890822", "imageRef": "bba331ba-faf5-4a56-81d9-6e84e9609665", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d5cd9b46-2491-4f10-bda2-40f7d635b974"}], "security_groups": [{"name": "tempest-securitygroup--95729993"}], "key_name": "tempest-keypair-1933428633", "user_data": "}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:58:32.480525 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.network.neutron [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] validate_networks() for [('d5cd9b46-2491-4f10-bda2-40f7d635b974', None, None, None, None, None)] {{(pid=130396) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 08:58:32.867930 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Flavor limits 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 08:58:32.868385 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Image limits 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 08:58:32.868857 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Flavor pref 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 08:58:32.869255 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Image pref 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 08:58:32.869657 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 08:58:32.879443 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:32.879946 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:32.880429 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:32.881232 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:32.881755 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:32.882222 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:32.899150 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-684d799b-6bf1-41c8-a002-10329c571a56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:32.899690 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Getting quotas for project 45be9d45ba4c4a2ca1ed053a7263dbee. Resources: {'ram', 'instances', 'cores'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:58:32.899907 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-684d799b-6bf1-41c8-a002-10329c571a56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:32.903248 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Getting quotas for user 94169e17266b4e6fad7e068a4c1384c7 and project 45be9d45ba4c4a2ca1ed053a7263dbee. Resources: {'ram', 'instances', 'cores'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:58:32.906024 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-684d799b-6bf1-41c8-a002-10329c571a56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:32.906506 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-684d799b-6bf1-41c8-a002-10329c571a56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:32.907497 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-684d799b-6bf1-41c8-a002-10329c571a56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:32.907823 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-684d799b-6bf1-41c8-a002-10329c571a56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:32.908316 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-684d799b-6bf1-41c8-a002-10329c571a56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:32.908738 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-684d799b-6bf1-41c8-a002-10329c571a56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:32.914669 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Going to run 1 instances... {{(pid=130396) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 08:58:32.917678 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bba331ba-faf5-4a56-81d9-6e84e9609665',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 08:58:32.975090 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-684d799b-6bf1-41c8-a002-10329c571a56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:32.978326 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-eb3a2872-9484-414a-aa9e-ccaee6f00f00 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.1 time: 0.610425 Oct 24 08:58:32.979069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 349/704] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 08:58:32 2022] POST /compute/v2.1/servers => generated 400 bytes in 613 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 08:58:32.995659 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-354d6199-3f59-4fbb-90f9-79a7cf348825 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:32.996883 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-354d6199-3f59-4fbb-90f9-79a7cf348825 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:33.003939 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-684d799b-6bf1-41c8-a002-10329c571a56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1271 microversion: 2.1 time: 0.108414 Oct 24 08:58:33.004837 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 356/705] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:32 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1271 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:58:33.095688 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-354d6199-3f59-4fbb-90f9-79a7cf348825 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:33.107001 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-354d6199-3f59-4fbb-90f9-79a7cf348825 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1237 microversion: 2.1 time: 0.119830 Oct 24 08:58:33.109265 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 350/706] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:32 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1237 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:58:34.046500 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7d3ffc03-9f9a-4e74-a831-f31cadb26168 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:34.047121 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7d3ffc03-9f9a-4e74-a831-f31cadb26168 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:34.054238 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7d3ffc03-9f9a-4e74-a831-f31cadb26168 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:34.054728 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7d3ffc03-9f9a-4e74-a831-f31cadb26168 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:34.055167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7d3ffc03-9f9a-4e74-a831-f31cadb26168 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:34.055876 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7d3ffc03-9f9a-4e74-a831-f31cadb26168 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:34.056368 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7d3ffc03-9f9a-4e74-a831-f31cadb26168 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:34.056795 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7d3ffc03-9f9a-4e74-a831-f31cadb26168 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:34.133531 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b0473217-61eb-4059-98c1-87779c47be60 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:34.134133 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b0473217-61eb-4059-98c1-87779c47be60 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:34.140069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0473217-61eb-4059-98c1-87779c47be60 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:34.140593 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0473217-61eb-4059-98c1-87779c47be60 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:34.142139 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0473217-61eb-4059-98c1-87779c47be60 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:34.142972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0473217-61eb-4059-98c1-87779c47be60 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:34.143447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0473217-61eb-4059-98c1-87779c47be60 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:34.143877 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0473217-61eb-4059-98c1-87779c47be60 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:34.235646 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b0473217-61eb-4059-98c1-87779c47be60 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:34.244330 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b0473217-61eb-4059-98c1-87779c47be60 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1289 microversion: 2.1 time: 0.116058 Oct 24 08:58:34.245367 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 351/707] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:34 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1289 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:58:34.307040 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7d3ffc03-9f9a-4e74-a831-f31cadb26168 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:34.315320 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7d3ffc03-9f9a-4e74-a831-f31cadb26168 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1319 microversion: 2.1 time: 0.272599 Oct 24 08:58:34.329077 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 357/708] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:34 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1319 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:34.683073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-6d956728-bb4e-46a5-b772-bcdf7db26f10 req-6ce17261-edee-4ecd-beb8-3362151266e2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e7ae65b9-4038-4fad-b243-bbf157408181", "tag": "ca181843-9f26-4a98-bbf5-490b90538167"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:58:34.692910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-6d956728-bb4e-46a5-b772-bcdf7db26f10 req-6ce17261-edee-4ecd-beb8-3362151266e2 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:34.693725 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-6d956728-bb4e-46a5-b772-bcdf7db26f10 req-6ce17261-edee-4ecd-beb8-3362151266e2 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:34.694361 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-6d956728-bb4e-46a5-b772-bcdf7db26f10 req-6ce17261-edee-4ecd-beb8-3362151266e2 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:34.707890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-6d956728-bb4e-46a5-b772-bcdf7db26f10 req-6ce17261-edee-4ecd-beb8-3362151266e2 service nova] Creating event network-changed:ca181843-9f26-4a98-bbf5-490b90538167 for instance e7ae65b9-4038-4fad-b243-bbf157408181 on hv-cinder-maste Oct 24 08:58:34.713682 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-6d956728-bb4e-46a5-b772-bcdf7db26f10 req-6ce17261-edee-4ecd-beb8-3362151266e2 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034244 Oct 24 08:58:34.714652 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 352/709] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:58:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:58:35.266969 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-957458ab-9e13-42c3-bf00-f2efca0ed55e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:35.267578 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-957458ab-9e13-42c3-bf00-f2efca0ed55e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:35.273634 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-957458ab-9e13-42c3-bf00-f2efca0ed55e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:35.274307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-957458ab-9e13-42c3-bf00-f2efca0ed55e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:35.274755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-957458ab-9e13-42c3-bf00-f2efca0ed55e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:35.275468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-957458ab-9e13-42c3-bf00-f2efca0ed55e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:35.275970 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-957458ab-9e13-42c3-bf00-f2efca0ed55e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:35.276426 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-957458ab-9e13-42c3-bf00-f2efca0ed55e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:35.348560 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-3b8c4379-96a7-480c-a35c-c315f0f3bfd5 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:35.350033 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-3b8c4379-96a7-480c-a35c-c315f0f3bfd5 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:35.364076 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3b8c4379-96a7-480c-a35c-c315f0f3bfd5 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:35.364607 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3b8c4379-96a7-480c-a35c-c315f0f3bfd5 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:35.365060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3b8c4379-96a7-480c-a35c-c315f0f3bfd5 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:35.365771 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3b8c4379-96a7-480c-a35c-c315f0f3bfd5 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:35.366222 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3b8c4379-96a7-480c-a35c-c315f0f3bfd5 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:35.371574 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3b8c4379-96a7-480c-a35c-c315f0f3bfd5 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:35.420389 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-957458ab-9e13-42c3-bf00-f2efca0ed55e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:35.428478 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-957458ab-9e13-42c3-bf00-f2efca0ed55e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1361 microversion: 2.1 time: 0.165342 Oct 24 08:58:35.430832 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 358/710] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:35 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1361 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:35.521582 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-3b8c4379-96a7-480c-a35c-c315f0f3bfd5 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:35.529379 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-3b8c4379-96a7-480c-a35c-c315f0f3bfd5 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1319 microversion: 2.1 time: 0.194063 Oct 24 08:58:35.529959 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 353/711] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:35 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1319 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:35.852055 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-0fdef658-4ee8-4241-92d7-ebdcacf66a01 req-3bd3f338-aa6f-471d-81ad-ee771e0bbc39 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5655b4d3-6c0f-4f8d-975d-024824e4a4b3", "tag": "3cf51600-b21d-4989-92db-8e04b3392067"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:58:35.862904 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-0fdef658-4ee8-4241-92d7-ebdcacf66a01 req-3bd3f338-aa6f-471d-81ad-ee771e0bbc39 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:35.863391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-0fdef658-4ee8-4241-92d7-ebdcacf66a01 req-3bd3f338-aa6f-471d-81ad-ee771e0bbc39 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:35.864082 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-0fdef658-4ee8-4241-92d7-ebdcacf66a01 req-3bd3f338-aa6f-471d-81ad-ee771e0bbc39 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:35.876422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-0fdef658-4ee8-4241-92d7-ebdcacf66a01 req-3bd3f338-aa6f-471d-81ad-ee771e0bbc39 service nova] Creating event network-changed:3cf51600-b21d-4989-92db-8e04b3392067 for instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 on hv-cinder-maste Oct 24 08:58:35.881301 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-0fdef658-4ee8-4241-92d7-ebdcacf66a01 req-3bd3f338-aa6f-471d-81ad-ee771e0bbc39 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032896 Oct 24 08:58:35.882050 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 359/712] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:58:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:58:36.450921 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b0d6c4ea-0dc6-453a-a5fe-f9eae597ec1f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:36.451588 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b0d6c4ea-0dc6-453a-a5fe-f9eae597ec1f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:36.457934 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0d6c4ea-0dc6-453a-a5fe-f9eae597ec1f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:36.458420 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0d6c4ea-0dc6-453a-a5fe-f9eae597ec1f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:36.458858 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0d6c4ea-0dc6-453a-a5fe-f9eae597ec1f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:36.459551 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0d6c4ea-0dc6-453a-a5fe-f9eae597ec1f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:36.460006 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0d6c4ea-0dc6-453a-a5fe-f9eae597ec1f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:36.460463 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0d6c4ea-0dc6-453a-a5fe-f9eae597ec1f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:36.554714 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a0dc789c-0103-4cf2-9fbb-23ec5aa89648 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:36.555413 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a0dc789c-0103-4cf2-9fbb-23ec5aa89648 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:36.561432 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0dc789c-0103-4cf2-9fbb-23ec5aa89648 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:36.561920 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0dc789c-0103-4cf2-9fbb-23ec5aa89648 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:36.562354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0dc789c-0103-4cf2-9fbb-23ec5aa89648 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:36.563271 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0dc789c-0103-4cf2-9fbb-23ec5aa89648 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:36.563806 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0dc789c-0103-4cf2-9fbb-23ec5aa89648 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:36.564102 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0dc789c-0103-4cf2-9fbb-23ec5aa89648 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:36.594862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b0d6c4ea-0dc6-453a-a5fe-f9eae597ec1f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:36.602939 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b0d6c4ea-0dc6-453a-a5fe-f9eae597ec1f tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1361 microversion: 2.1 time: 0.155939 Oct 24 08:58:36.603526 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 354/713] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:36 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1361 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:36.704107 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a0dc789c-0103-4cf2-9fbb-23ec5aa89648 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:36.711802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a0dc789c-0103-4cf2-9fbb-23ec5aa89648 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1319 microversion: 2.1 time: 0.161664 Oct 24 08:58:36.712496 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 360/714] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:36 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1319 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:58:37.627650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5554dc08-ee86-4bb5-b21f-5459d60b596c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:37.628831 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-5554dc08-ee86-4bb5-b21f-5459d60b596c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:37.635183 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5554dc08-ee86-4bb5-b21f-5459d60b596c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:37.635699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5554dc08-ee86-4bb5-b21f-5459d60b596c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:37.636158 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5554dc08-ee86-4bb5-b21f-5459d60b596c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:37.636871 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5554dc08-ee86-4bb5-b21f-5459d60b596c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:37.637356 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5554dc08-ee86-4bb5-b21f-5459d60b596c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:37.637790 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5554dc08-ee86-4bb5-b21f-5459d60b596c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:37.735193 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e7dea9fc-5293-40b9-81e8-65af0e6382c4 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:37.735876 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e7dea9fc-5293-40b9-81e8-65af0e6382c4 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:37.742807 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7dea9fc-5293-40b9-81e8-65af0e6382c4 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:37.743302 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7dea9fc-5293-40b9-81e8-65af0e6382c4 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:37.743744 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7dea9fc-5293-40b9-81e8-65af0e6382c4 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:37.744491 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7dea9fc-5293-40b9-81e8-65af0e6382c4 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:37.744987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7dea9fc-5293-40b9-81e8-65af0e6382c4 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:37.745375 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e7dea9fc-5293-40b9-81e8-65af0e6382c4 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:37.905378 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5554dc08-ee86-4bb5-b21f-5459d60b596c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:37.924763 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5554dc08-ee86-4bb5-b21f-5459d60b596c tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1361 microversion: 2.1 time: 0.301482 Oct 24 08:58:37.926098 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 355/715] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:37 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1361 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:37.944763 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e7dea9fc-5293-40b9-81e8-65af0e6382c4 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:37.952682 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e7dea9fc-5293-40b9-81e8-65af0e6382c4 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1319 microversion: 2.1 time: 0.221665 Oct 24 08:58:37.954251 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 361/716] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:37 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1319 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:38.948580 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-792c47af-2432-4b5b-a0e1-9dd074ba65a3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:38.949192 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-792c47af-2432-4b5b-a0e1-9dd074ba65a3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:38.956190 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-792c47af-2432-4b5b-a0e1-9dd074ba65a3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:38.956695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-792c47af-2432-4b5b-a0e1-9dd074ba65a3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:38.957139 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-792c47af-2432-4b5b-a0e1-9dd074ba65a3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:38.957838 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-792c47af-2432-4b5b-a0e1-9dd074ba65a3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:38.958296 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-792c47af-2432-4b5b-a0e1-9dd074ba65a3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:38.958722 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-792c47af-2432-4b5b-a0e1-9dd074ba65a3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:38.976554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-3ffa6d6b-fae0-4174-9b84-0c0517b8332d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:38.977236 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-3ffa6d6b-fae0-4174-9b84-0c0517b8332d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:38.983466 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ffa6d6b-fae0-4174-9b84-0c0517b8332d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:38.983952 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ffa6d6b-fae0-4174-9b84-0c0517b8332d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:38.984436 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ffa6d6b-fae0-4174-9b84-0c0517b8332d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:38.985347 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ffa6d6b-fae0-4174-9b84-0c0517b8332d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:38.985869 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ffa6d6b-fae0-4174-9b84-0c0517b8332d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:38.986842 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ffa6d6b-fae0-4174-9b84-0c0517b8332d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:39.161346 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-792c47af-2432-4b5b-a0e1-9dd074ba65a3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:39.168374 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-3ffa6d6b-fae0-4174-9b84-0c0517b8332d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:39.169137 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-792c47af-2432-4b5b-a0e1-9dd074ba65a3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1361 microversion: 2.1 time: 0.224360 Oct 24 08:58:39.169727 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 356/717] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:38 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1361 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:39.176369 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-3ffa6d6b-fae0-4174-9b84-0c0517b8332d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1319 microversion: 2.1 time: 0.203677 Oct 24 08:58:39.176935 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 362/718] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:38 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1319 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:40.192734 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-228eb613-8832-4094-a546-328b35678990 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:40.193396 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-228eb613-8832-4094-a546-328b35678990 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:40.199488 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f9afab14-7641-4475-b872-6921d17ec34f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:40.200103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f9afab14-7641-4475-b872-6921d17ec34f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:40.200354 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-228eb613-8832-4094-a546-328b35678990 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:40.200829 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-228eb613-8832-4094-a546-328b35678990 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:40.201272 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-228eb613-8832-4094-a546-328b35678990 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:40.201986 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-228eb613-8832-4094-a546-328b35678990 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:40.202447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-228eb613-8832-4094-a546-328b35678990 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:40.202954 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-228eb613-8832-4094-a546-328b35678990 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:40.208053 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f9afab14-7641-4475-b872-6921d17ec34f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:40.208568 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f9afab14-7641-4475-b872-6921d17ec34f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:40.209001 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f9afab14-7641-4475-b872-6921d17ec34f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:40.209925 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f9afab14-7641-4475-b872-6921d17ec34f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:40.210598 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f9afab14-7641-4475-b872-6921d17ec34f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:40.211027 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f9afab14-7641-4475-b872-6921d17ec34f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:40.404404 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-228eb613-8832-4094-a546-328b35678990 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:40.408749 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f9afab14-7641-4475-b872-6921d17ec34f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:40.412899 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-228eb613-8832-4094-a546-328b35678990 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1361 microversion: 2.1 time: 0.225140 Oct 24 08:58:40.413489 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 357/719] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:40 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1361 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:58:40.416433 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f9afab14-7641-4475-b872-6921d17ec34f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1319 microversion: 2.1 time: 0.221444 Oct 24 08:58:40.417150 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 363/720] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:40 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1319 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:40.419858 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-90ed6472-5851-48fc-a121-85200a645d5e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:40.420378 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-90ed6472-5851-48fc-a121-85200a645d5e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:40.426271 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-90ed6472-5851-48fc-a121-85200a645d5e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:40.426744 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-90ed6472-5851-48fc-a121-85200a645d5e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:40.427179 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-90ed6472-5851-48fc-a121-85200a645d5e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:40.427871 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-90ed6472-5851-48fc-a121-85200a645d5e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:40.428354 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-90ed6472-5851-48fc-a121-85200a645d5e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:40.428902 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-90ed6472-5851-48fc-a121-85200a645d5e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:40.451866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-90ed6472-5851-48fc-a121-85200a645d5e tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf/ips" status: 200 len: 150 microversion: 2.1 time: 0.037238 Oct 24 08:58:40.452472 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 358/721] 10.222.0.39 () {58 vars in 1248 bytes} [Mon Oct 24 08:58:40 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf/ips => generated 150 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:58:41.436317 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7be86205-e5fe-43de-8580-a2b4b6fdd5dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:41.437113 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7be86205-e5fe-43de-8580-a2b4b6fdd5dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:41.439398 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-73b2d248-2e6f-4031-9485-d2a55efdb914 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:41.440015 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-73b2d248-2e6f-4031-9485-d2a55efdb914 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:41.443853 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7be86205-e5fe-43de-8580-a2b4b6fdd5dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:41.444382 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7be86205-e5fe-43de-8580-a2b4b6fdd5dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:41.444821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7be86205-e5fe-43de-8580-a2b4b6fdd5dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:41.445531 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7be86205-e5fe-43de-8580-a2b4b6fdd5dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:41.445979 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7be86205-e5fe-43de-8580-a2b4b6fdd5dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:41.446405 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7be86205-e5fe-43de-8580-a2b4b6fdd5dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:41.446529 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-73b2d248-2e6f-4031-9485-d2a55efdb914 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:41.446907 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-73b2d248-2e6f-4031-9485-d2a55efdb914 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:41.447341 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-73b2d248-2e6f-4031-9485-d2a55efdb914 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:41.448036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-73b2d248-2e6f-4031-9485-d2a55efdb914 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:41.448518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-73b2d248-2e6f-4031-9485-d2a55efdb914 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:41.448947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-73b2d248-2e6f-4031-9485-d2a55efdb914 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:41.643178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7be86205-e5fe-43de-8580-a2b4b6fdd5dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:41.650592 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-73b2d248-2e6f-4031-9485-d2a55efdb914 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:41.651014 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7be86205-e5fe-43de-8580-a2b4b6fdd5dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1361 microversion: 2.1 time: 0.218901 Oct 24 08:58:41.651602 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 364/722] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:41 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1361 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:41.659706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-73b2d248-2e6f-4031-9485-d2a55efdb914 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1319 microversion: 2.1 time: 0.224693 Oct 24 08:58:41.660385 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 359/723] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:41 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1319 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:58:41.666082 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-68f1321e-a77d-41c8-b2a7-89f017df1c8f req-a43eb19c-ac05-4210-91a5-15d2c4665e6d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e7ae65b9-4038-4fad-b243-bbf157408181", "name": "network-vif-plugged", "status": "completed", "tag": "ca181843-9f26-4a98-bbf5-490b90538167"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:58:41.678857 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-68f1321e-a77d-41c8-b2a7-89f017df1c8f req-a43eb19c-ac05-4210-91a5-15d2c4665e6d service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:41.679337 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-68f1321e-a77d-41c8-b2a7-89f017df1c8f req-a43eb19c-ac05-4210-91a5-15d2c4665e6d service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:41.679764 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-68f1321e-a77d-41c8-b2a7-89f017df1c8f req-a43eb19c-ac05-4210-91a5-15d2c4665e6d service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:41.693578 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-68f1321e-a77d-41c8-b2a7-89f017df1c8f req-a43eb19c-ac05-4210-91a5-15d2c4665e6d service nova] Creating event network-vif-plugged:ca181843-9f26-4a98-bbf5-490b90538167 for instance e7ae65b9-4038-4fad-b243-bbf157408181 on hv-cinder-maste Oct 24 08:58:41.698367 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-68f1321e-a77d-41c8-b2a7-89f017df1c8f req-a43eb19c-ac05-4210-91a5-15d2c4665e6d service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036479 Oct 24 08:58:41.699129 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 365/724] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:58:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:58:41.791371 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-57f7faf6-0987-49cf-9b90-ecb400c42ea8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "399a99f3-6f1b-428b-9b75-deff80343276"}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:58:41.793848 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-57f7faf6-0987-49cf-9b90-ecb400c42ea8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:41.799873 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-57f7faf6-0987-49cf-9b90-ecb400c42ea8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:41.800395 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-57f7faf6-0987-49cf-9b90-ecb400c42ea8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:41.800838 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-57f7faf6-0987-49cf-9b90-ecb400c42ea8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:41.801540 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-57f7faf6-0987-49cf-9b90-ecb400c42ea8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:41.801993 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-57f7faf6-0987-49cf-9b90-ecb400c42ea8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:41.802419 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-57f7faf6-0987-49cf-9b90-ecb400c42ea8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:42.079499 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-57f7faf6-0987-49cf-9b90-ecb400c42ea8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "POST /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.291890 Oct 24 08:58:42.080113 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 360/725] 10.222.0.39 () {60 vars in 1323 bytes} [Mon Oct 24 08:58:41 2022] POST /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf/os-volume_attachments => generated 194 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:58:42.675709 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8bf6cc90-02b0-47b5-9ee1-dd1a361dafd7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:42.676440 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8bf6cc90-02b0-47b5-9ee1-dd1a361dafd7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:42.678463 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=6 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:58:42.689311 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-3f41343f-e4b2-4406-b82e-9ca9fdf15bd1 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:42.690025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-3f41343f-e4b2-4406-b82e-9ca9fdf15bd1 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:42.690161 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8bf6cc90-02b0-47b5-9ee1-dd1a361dafd7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:42.690579 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8bf6cc90-02b0-47b5-9ee1-dd1a361dafd7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:42.691017 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8bf6cc90-02b0-47b5-9ee1-dd1a361dafd7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:42.691721 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8bf6cc90-02b0-47b5-9ee1-dd1a361dafd7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:42.692194 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8bf6cc90-02b0-47b5-9ee1-dd1a361dafd7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:42.692621 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8bf6cc90-02b0-47b5-9ee1-dd1a361dafd7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:42.697349 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3f41343f-e4b2-4406-b82e-9ca9fdf15bd1 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:42.697828 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3f41343f-e4b2-4406-b82e-9ca9fdf15bd1 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:42.698294 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3f41343f-e4b2-4406-b82e-9ca9fdf15bd1 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:42.698960 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3f41343f-e4b2-4406-b82e-9ca9fdf15bd1 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:42.699404 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3f41343f-e4b2-4406-b82e-9ca9fdf15bd1 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:42.699823 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3f41343f-e4b2-4406-b82e-9ca9fdf15bd1 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:42.889080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8bf6cc90-02b0-47b5-9ee1-dd1a361dafd7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:42.892866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-3f41343f-e4b2-4406-b82e-9ca9fdf15bd1 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:42.895143 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=4 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:58:42.898061 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8bf6cc90-02b0-47b5-9ee1-dd1a361dafd7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1361 microversion: 2.1 time: 0.226743 Oct 24 08:58:42.898649 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 366/726] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:42 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1361 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:42.902237 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-3f41343f-e4b2-4406-b82e-9ca9fdf15bd1 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1319 microversion: 2.1 time: 0.223334 Oct 24 08:58:42.902791 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 361/727] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:42 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1319 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:58:43.924530 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-cfb2ecc8-a093-4b57-836b-be79db188013 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:43.925111 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-cfb2ecc8-a093-4b57-836b-be79db188013 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:43.926391 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c4f4887e-1dc1-41ca-b474-1ac80cd75022 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:43.927015 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c4f4887e-1dc1-41ca-b474-1ac80cd75022 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:43.929471 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=14 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:58:43.931821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cfb2ecc8-a093-4b57-836b-be79db188013 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:43.932340 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cfb2ecc8-a093-4b57-836b-be79db188013 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:43.932786 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cfb2ecc8-a093-4b57-836b-be79db188013 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:43.933471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cfb2ecc8-a093-4b57-836b-be79db188013 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:43.934062 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cfb2ecc8-a093-4b57-836b-be79db188013 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:43.934481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cfb2ecc8-a093-4b57-836b-be79db188013 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:43.934640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c4f4887e-1dc1-41ca-b474-1ac80cd75022 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:43.935136 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c4f4887e-1dc1-41ca-b474-1ac80cd75022 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:43.935579 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c4f4887e-1dc1-41ca-b474-1ac80cd75022 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:43.936301 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c4f4887e-1dc1-41ca-b474-1ac80cd75022 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:43.936793 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c4f4887e-1dc1-41ca-b474-1ac80cd75022 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:43.937221 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c4f4887e-1dc1-41ca-b474-1ac80cd75022 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:43.941876 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:58:44.147688 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c4f4887e-1dc1-41ca-b474-1ac80cd75022 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:44.151009 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cfb2ecc8-a093-4b57-836b-be79db188013 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:44.155854 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c4f4887e-1dc1-41ca-b474-1ac80cd75022 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1509 microversion: 2.1 time: 0.234259 Oct 24 08:58:44.159174 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 362/728] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:43 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1509 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:58:44.160741 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-cfb2ecc8-a093-4b57-836b-be79db188013 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1361 microversion: 2.1 time: 0.241477 Oct 24 08:58:44.163074 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-432ae10b-ce7e-47d1-940c-2d010a9e6077 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:44.164427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-432ae10b-ce7e-47d1-940c-2d010a9e6077 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:44.170908 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-432ae10b-ce7e-47d1-940c-2d010a9e6077 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:44.171585 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-432ae10b-ce7e-47d1-940c-2d010a9e6077 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:44.172431 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-432ae10b-ce7e-47d1-940c-2d010a9e6077 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:44.173108 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 367/729] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:43 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1361 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:44.173645 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-432ae10b-ce7e-47d1-940c-2d010a9e6077 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:44.174497 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-432ae10b-ce7e-47d1-940c-2d010a9e6077 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:44.175155 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-432ae10b-ce7e-47d1-940c-2d010a9e6077 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:44.205229 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-432ae10b-ce7e-47d1-940c-2d010a9e6077 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3/ips" status: 200 len: 151 microversion: 2.20 time: 0.046113 Oct 24 08:58:44.206101 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 363/730] 10.222.0.39 () {60 vars in 1289 bytes} [Mon Oct 24 08:58:44 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3/ips => generated 151 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:58:45.184500 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b38d4f7b-7985-42a3-8bb3-a4b45eefc983 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:45.185052 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b38d4f7b-7985-42a3-8bb3-a4b45eefc983 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:45.192891 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b38d4f7b-7985-42a3-8bb3-a4b45eefc983 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:45.193407 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b38d4f7b-7985-42a3-8bb3-a4b45eefc983 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:45.193856 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b38d4f7b-7985-42a3-8bb3-a4b45eefc983 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:45.194591 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b38d4f7b-7985-42a3-8bb3-a4b45eefc983 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:45.195051 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b38d4f7b-7985-42a3-8bb3-a4b45eefc983 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:45.195479 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b38d4f7b-7985-42a3-8bb3-a4b45eefc983 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:45.339194 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-b38d4f7b-7985-42a3-8bb3-a4b45eefc983 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:45.349320 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b38d4f7b-7985-42a3-8bb3-a4b45eefc983 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1361 microversion: 2.1 time: 0.168447 Oct 24 08:58:45.349920 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 368/731] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:45 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1361 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:45.683681 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-f086d63c-55dc-46c5-a236-a072f336f54e req-a343c131-69e3-452c-a5ea-843e11777ded service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5655b4d3-6c0f-4f8d-975d-024824e4a4b3", "name": "network-vif-plugged", "status": "completed", "tag": "3cf51600-b21d-4989-92db-8e04b3392067"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:58:45.697232 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-f086d63c-55dc-46c5-a236-a072f336f54e req-a343c131-69e3-452c-a5ea-843e11777ded service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:45.697974 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-f086d63c-55dc-46c5-a236-a072f336f54e req-a343c131-69e3-452c-a5ea-843e11777ded service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:45.698418 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-f086d63c-55dc-46c5-a236-a072f336f54e req-a343c131-69e3-452c-a5ea-843e11777ded service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:45.710768 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-f086d63c-55dc-46c5-a236-a072f336f54e req-a343c131-69e3-452c-a5ea-843e11777ded service nova] Creating event network-vif-plugged:3cf51600-b21d-4989-92db-8e04b3392067 for instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 on hv-cinder-maste Oct 24 08:58:45.715765 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-f086d63c-55dc-46c5-a236-a072f336f54e req-a343c131-69e3-452c-a5ea-843e11777ded service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036267 Oct 24 08:58:45.716938 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 364/732] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:58:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:58:46.206254 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a83907d2-d3ad-4492-8f5e-c464d0ee6ae5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:58:46.206795 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a83907d2-d3ad-4492-8f5e-c464d0ee6ae5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:46.213188 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a83907d2-d3ad-4492-8f5e-c464d0ee6ae5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:46.213662 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a83907d2-d3ad-4492-8f5e-c464d0ee6ae5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:46.214099 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a83907d2-d3ad-4492-8f5e-c464d0ee6ae5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:46.214826 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a83907d2-d3ad-4492-8f5e-c464d0ee6ae5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:46.215285 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a83907d2-d3ad-4492-8f5e-c464d0ee6ae5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:46.215708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a83907d2-d3ad-4492-8f5e-c464d0ee6ae5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:46.234844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:58:46.241193 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-a83907d2-d3ad-4492-8f5e-c464d0ee6ae5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lazy-loading 'flavor' on Instance uuid e72acc7a-decb-46c7-9c51-f731e2c413b3 {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:58:46.388948 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e3ab4319-c7d2-4d65-ac28-47747f2d7a72 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:46.389657 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e3ab4319-c7d2-4d65-ac28-47747f2d7a72 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:46.396598 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e3ab4319-c7d2-4d65-ac28-47747f2d7a72 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:46.397113 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e3ab4319-c7d2-4d65-ac28-47747f2d7a72 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:46.397516 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e3ab4319-c7d2-4d65-ac28-47747f2d7a72 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:46.398554 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e3ab4319-c7d2-4d65-ac28-47747f2d7a72 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:46.399016 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e3ab4319-c7d2-4d65-ac28-47747f2d7a72 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:46.399440 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e3ab4319-c7d2-4d65-ac28-47747f2d7a72 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:46.446380 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a83907d2-d3ad-4492-8f5e-c464d0ee6ae5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "POST /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3/action" status: 202 len: 0 microversion: 2.20 time: 0.244663 Oct 24 08:58:46.446989 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 369/733] 10.222.0.39 () {62 vars in 1319 bytes} [Mon Oct 24 08:58:46 2022] POST /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3/action => generated 0 bytes in 246 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 24 08:58:46.455883 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f45e40f7-c037-430e-8e61-809a6ff54640 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:46.456675 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f45e40f7-c037-430e-8e61-809a6ff54640 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:46.462914 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f45e40f7-c037-430e-8e61-809a6ff54640 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:46.463398 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f45e40f7-c037-430e-8e61-809a6ff54640 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:46.463835 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f45e40f7-c037-430e-8e61-809a6ff54640 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:46.464558 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f45e40f7-c037-430e-8e61-809a6ff54640 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:46.465019 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f45e40f7-c037-430e-8e61-809a6ff54640 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:46.465438 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f45e40f7-c037-430e-8e61-809a6ff54640 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:46.579695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e3ab4319-c7d2-4d65-ac28-47747f2d7a72 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:46.587663 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e3ab4319-c7d2-4d65-ac28-47747f2d7a72 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1361 microversion: 2.1 time: 0.217641 Oct 24 08:58:46.588262 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 365/734] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:46 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1361 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:58:46.628522 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f45e40f7-c037-430e-8e61-809a6ff54640 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:46.638294 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f45e40f7-c037-430e-8e61-809a6ff54640 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:46.639848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f45e40f7-c037-430e-8e61-809a6ff54640 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:46.641162 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f45e40f7-c037-430e-8e61-809a6ff54640 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1723 microversion: 2.20 time: 0.189798 Oct 24 08:58:46.648280 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 370/735] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:58:46 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1723 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:58:47.610817 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6c2c8abb-a94e-4f54-9985-14ad1edba156 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:47.611362 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6c2c8abb-a94e-4f54-9985-14ad1edba156 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:47.618353 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6c2c8abb-a94e-4f54-9985-14ad1edba156 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:47.618864 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6c2c8abb-a94e-4f54-9985-14ad1edba156 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:47.619268 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6c2c8abb-a94e-4f54-9985-14ad1edba156 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:47.619968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6c2c8abb-a94e-4f54-9985-14ad1edba156 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:47.620454 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6c2c8abb-a94e-4f54-9985-14ad1edba156 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:47.620877 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6c2c8abb-a94e-4f54-9985-14ad1edba156 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:47.667662 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-987aba82-1284-41d7-b8c3-0b83b4d19b74 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:47.668280 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-987aba82-1284-41d7-b8c3-0b83b4d19b74 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:47.674755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-987aba82-1284-41d7-b8c3-0b83b4d19b74 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:47.675256 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-987aba82-1284-41d7-b8c3-0b83b4d19b74 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:47.675687 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-987aba82-1284-41d7-b8c3-0b83b4d19b74 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:47.676485 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-987aba82-1284-41d7-b8c3-0b83b4d19b74 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:47.676876 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-987aba82-1284-41d7-b8c3-0b83b4d19b74 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:47.677308 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-987aba82-1284-41d7-b8c3-0b83b4d19b74 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:47.806893 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6c2c8abb-a94e-4f54-9985-14ad1edba156 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:47.816975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6c2c8abb-a94e-4f54-9985-14ad1edba156 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1361 microversion: 2.1 time: 0.210037 Oct 24 08:58:47.829624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 366/736] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:47 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1361 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:47.854981 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-987aba82-1284-41d7-b8c3-0b83b4d19b74 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:47.864285 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-987aba82-1284-41d7-b8c3-0b83b4d19b74 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:47.865677 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-987aba82-1284-41d7-b8c3-0b83b4d19b74 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:47.866873 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-987aba82-1284-41d7-b8c3-0b83b4d19b74 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1723 microversion: 2.20 time: 0.203115 Oct 24 08:58:47.867434 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 371/737] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:58:47 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1723 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:58:48.842025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-54a1bcad-7dac-4abb-97eb-aaec2daee6bb tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:48.842728 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-54a1bcad-7dac-4abb-97eb-aaec2daee6bb tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:48.849316 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-54a1bcad-7dac-4abb-97eb-aaec2daee6bb tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:48.849739 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-54a1bcad-7dac-4abb-97eb-aaec2daee6bb tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:48.850172 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-54a1bcad-7dac-4abb-97eb-aaec2daee6bb tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:48.850866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-54a1bcad-7dac-4abb-97eb-aaec2daee6bb tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:48.851318 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-54a1bcad-7dac-4abb-97eb-aaec2daee6bb tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:48.851741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-54a1bcad-7dac-4abb-97eb-aaec2daee6bb tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:48.893735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-aa0ef70d-7e6e-48da-ba82-2e1c3cbedc49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:48.894343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-aa0ef70d-7e6e-48da-ba82-2e1c3cbedc49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:48.903177 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa0ef70d-7e6e-48da-ba82-2e1c3cbedc49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:48.903658 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa0ef70d-7e6e-48da-ba82-2e1c3cbedc49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:48.904091 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa0ef70d-7e6e-48da-ba82-2e1c3cbedc49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:48.904817 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa0ef70d-7e6e-48da-ba82-2e1c3cbedc49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:48.905261 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa0ef70d-7e6e-48da-ba82-2e1c3cbedc49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:48.905678 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa0ef70d-7e6e-48da-ba82-2e1c3cbedc49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:49.043530 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-54a1bcad-7dac-4abb-97eb-aaec2daee6bb tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:49.055232 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-54a1bcad-7dac-4abb-97eb-aaec2daee6bb tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1546 microversion: 2.1 time: 0.217036 Oct 24 08:58:49.068178 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 367/738] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:48 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1546 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:49.074945 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-aa0ef70d-7e6e-48da-ba82-2e1c3cbedc49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:49.077813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c0c35a79-2145-4ef3-8985-5191ab5ceb39 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:49.078545 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c0c35a79-2145-4ef3-8985-5191ab5ceb39 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:49.083968 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-aa0ef70d-7e6e-48da-ba82-2e1c3cbedc49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:49.084635 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0c35a79-2145-4ef3-8985-5191ab5ceb39 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:49.085131 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0c35a79-2145-4ef3-8985-5191ab5ceb39 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:49.085533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-aa0ef70d-7e6e-48da-ba82-2e1c3cbedc49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:49.085652 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0c35a79-2145-4ef3-8985-5191ab5ceb39 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:49.086258 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0c35a79-2145-4ef3-8985-5191ab5ceb39 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:49.086703 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-aa0ef70d-7e6e-48da-ba82-2e1c3cbedc49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1723 microversion: 2.20 time: 0.197613 Oct 24 08:58:49.086813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0c35a79-2145-4ef3-8985-5191ab5ceb39 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:49.087128 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0c35a79-2145-4ef3-8985-5191ab5ceb39 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:49.087430 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 372/739] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:58:48 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1723 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:58:49.167632 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c0c35a79-2145-4ef3-8985-5191ab5ceb39 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.093590 Oct 24 08:58:49.168264 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 368/740] 10.222.0.39 () {58 vars in 1275 bytes} [Mon Oct 24 08:58:49 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-interface => generated 284 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:58:50.114474 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-6d6a1549-2954-40f2-b102-144479ae769b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:50.115194 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-6d6a1549-2954-40f2-b102-144479ae769b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:50.121644 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d6a1549-2954-40f2-b102-144479ae769b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:50.122237 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d6a1549-2954-40f2-b102-144479ae769b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:50.122747 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d6a1549-2954-40f2-b102-144479ae769b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:50.123666 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d6a1549-2954-40f2-b102-144479ae769b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:50.124123 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d6a1549-2954-40f2-b102-144479ae769b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:50.124577 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d6a1549-2954-40f2-b102-144479ae769b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:50.341866 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-6d6a1549-2954-40f2-b102-144479ae769b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:50.351181 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-6d6a1549-2954-40f2-b102-144479ae769b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:50.352668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-6d6a1549-2954-40f2-b102-144479ae769b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:50.354015 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-6d6a1549-2954-40f2-b102-144479ae769b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1723 microversion: 2.20 time: 0.243891 Oct 24 08:58:50.354594 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 373/741] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:58:50 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1723 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:58:50.491414 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-8a7ca178-dba2-4a51-987d-e786919337bd req-beaefdd6-92d6-40c6-8a4b-2ca286b2772b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5655b4d3-6c0f-4f8d-975d-024824e4a4b3", "tag": "3cf51600-b21d-4989-92db-8e04b3392067"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:58:50.501580 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-8a7ca178-dba2-4a51-987d-e786919337bd req-beaefdd6-92d6-40c6-8a4b-2ca286b2772b service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:50.502056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-8a7ca178-dba2-4a51-987d-e786919337bd req-beaefdd6-92d6-40c6-8a4b-2ca286b2772b service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:50.502493 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-8a7ca178-dba2-4a51-987d-e786919337bd req-beaefdd6-92d6-40c6-8a4b-2ca286b2772b service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:50.513728 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-8a7ca178-dba2-4a51-987d-e786919337bd req-beaefdd6-92d6-40c6-8a4b-2ca286b2772b service nova] Creating event network-changed:3cf51600-b21d-4989-92db-8e04b3392067 for instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 on hv-cinder-maste Oct 24 08:58:50.518838 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-8a7ca178-dba2-4a51-987d-e786919337bd req-beaefdd6-92d6-40c6-8a4b-2ca286b2772b service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031910 Oct 24 08:58:50.519618 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 369/742] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:58:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:58:50.765517 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f4254780-ad7f-4083-8c5f-5405aee73555 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "399a99f3-6f1b-428b-9b75-deff80343276"}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:58:50.768902 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f4254780-ad7f-4083-8c5f-5405aee73555 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:50.775138 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f4254780-ad7f-4083-8c5f-5405aee73555 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:50.775627 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f4254780-ad7f-4083-8c5f-5405aee73555 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:50.776059 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f4254780-ad7f-4083-8c5f-5405aee73555 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:50.776800 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f4254780-ad7f-4083-8c5f-5405aee73555 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:50.777247 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f4254780-ad7f-4083-8c5f-5405aee73555 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:50.777669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f4254780-ad7f-4083-8c5f-5405aee73555 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:50.884863 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-f4254780-ad7f-4083-8c5f-5405aee73555 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] HTTP exception thrown: Invalid volume: volume 399a99f3-6f1b-428b-9b75-deff80343276 is already attached to instances: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f4254780-ad7f-4083-8c5f-5405aee73555 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Request method failure captured: Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: POST /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf/os-volume_attachments HTTP/1.1 Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 74 Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlH_P2cgqdGxTyt_DjhRKpesrNF98h-yweC7dRlZggOYZXXonk3ROJmezNObJqlbezl2V7Pqsl8fDKOVcqEGBVvIZd3l8-ZBbHliak_eivuyadmAWxfis-R_etsiC3KVVo1LvGlKxFu0GFkQBrTFC5gIfs64iGB92PfDe2tNtbCL85U Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 08:58:50.887066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 08:58:50.888567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b34f10ec378a4622a8f0b0546e84da7f", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}] Oct 24 08:58:50.888567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-AttachVolumeNegativeTest-97597731 Oct 24 08:58:50.888567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 08:58:50.888567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 08:58:50.888567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 08:58:50.888567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 08:58:50.888567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 08:58:50.888567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: ebdef1d69bb14837b9a0c28cbba332fc Oct 24 08:58:50.888567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 08:58:50.888567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Oct 24 08:58:50.888567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {"volumeAttachment": {"volumeId": "399a99f3-6f1b-428b-9b75-deff80343276"}} Oct 24 08:58:50.888567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Invalid volume: volume 399a99f3-6f1b-428b-9b75-deff80343276 is already attached to instances: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'server_id': 'd9301ca5-bdaa-4f5d-8bcd-772b8923b1cf', 'body': {'volumeAttachment': {'volumeId': '399a99f3-6f1b-428b-9b75-deff80343276'}}} Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 399a99f3-6f1b-428b-9b75-deff80343276 is already attached to instances: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:58:50.889378 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 399a99f3-6f1b-428b-9b75-deff80343276 is already attached to instances: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:58:50.890962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:58:50.892863 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:58:50.892863 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:58:50.892863 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:58:50.892863 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 399a99f3-6f1b-428b-9b75-deff80343276 is already attached to instances: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf Oct 24 08:58:50.892863 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 08:58:50.892863 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f4254780-ad7f-4083-8c5f-5405aee73555 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Returning 400 to user: Invalid volume: volume 399a99f3-6f1b-428b-9b75-deff80343276 is already attached to instances: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:58:50.892863 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f4254780-ad7f-4083-8c5f-5405aee73555 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "POST /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.132218 Oct 24 08:58:50.892863 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 374/743] 10.222.0.39 () {60 vars in 1323 bytes} [Mon Oct 24 08:58:50 2022] POST /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf/os-volume_attachments => generated 174 bytes in 134 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 24 08:58:50.972790 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-38a6ca9d-b837-40b4-a543-dacff23aaa77 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:50.973610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-38a6ca9d-b837-40b4-a543-dacff23aaa77 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:50.980680 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-38a6ca9d-b837-40b4-a543-dacff23aaa77 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:50.981425 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-38a6ca9d-b837-40b4-a543-dacff23aaa77 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:50.982012 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-38a6ca9d-b837-40b4-a543-dacff23aaa77 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:50.982883 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-38a6ca9d-b837-40b4-a543-dacff23aaa77 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:50.984054 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-38a6ca9d-b837-40b4-a543-dacff23aaa77 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:50.986371 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-38a6ca9d-b837-40b4-a543-dacff23aaa77 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:51.082986 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.objects.instance [None req-38a6ca9d-b837-40b4-a543-dacff23aaa77 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lazy-loading 'services' on Instance uuid d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf {{(pid=130396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:58:51.166223 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-38a6ca9d-b837-40b4-a543-dacff23aaa77 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "DELETE /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf/os-volume_attachments/399a99f3-6f1b-428b-9b75-deff80343276" status: 202 len: 0 microversion: 2.1 time: 0.201466 Oct 24 08:58:51.167045 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 370/744] 10.222.0.39 () {58 vars in 1416 bytes} [Mon Oct 24 08:58:50 2022] DELETE /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf/os-volume_attachments/399a99f3-6f1b-428b-9b75-deff80343276 => generated 0 bytes in 203 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 08:58:51.181188 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d8057e9d-7908-4066-9518-c09b61ac2730 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:51.190319 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d8057e9d-7908-4066-9518-c09b61ac2730 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:51.200405 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d8057e9d-7908-4066-9518-c09b61ac2730 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:51.201264 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d8057e9d-7908-4066-9518-c09b61ac2730 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:51.201955 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d8057e9d-7908-4066-9518-c09b61ac2730 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:51.203472 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d8057e9d-7908-4066-9518-c09b61ac2730 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:51.204200 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d8057e9d-7908-4066-9518-c09b61ac2730 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:51.204839 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d8057e9d-7908-4066-9518-c09b61ac2730 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:51.252074 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d8057e9d-7908-4066-9518-c09b61ac2730 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.080614 Oct 24 08:58:51.253143 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 375/745] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 08:58:51 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf/os-volume_attachments => generated 197 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:58:51.383290 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0a28e6a5-478a-41f7-ae7b-921347510754 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:51.384027 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-0a28e6a5-478a-41f7-ae7b-921347510754 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:51.391079 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a28e6a5-478a-41f7-ae7b-921347510754 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:51.392003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a28e6a5-478a-41f7-ae7b-921347510754 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:51.392124 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a28e6a5-478a-41f7-ae7b-921347510754 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:51.392765 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a28e6a5-478a-41f7-ae7b-921347510754 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:51.393223 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a28e6a5-478a-41f7-ae7b-921347510754 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:51.393648 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a28e6a5-478a-41f7-ae7b-921347510754 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:51.540661 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0a28e6a5-478a-41f7-ae7b-921347510754 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:51.549871 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0a28e6a5-478a-41f7-ae7b-921347510754 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:51.551315 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0a28e6a5-478a-41f7-ae7b-921347510754 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:51.552571 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-0a28e6a5-478a-41f7-ae7b-921347510754 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1723 microversion: 2.20 time: 0.173572 Oct 24 08:58:51.553122 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 371/746] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:58:51 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1723 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:58:52.264721 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b04adbcb-8e25-41f9-9364-88700ffae293 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:52.268043 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b04adbcb-8e25-41f9-9364-88700ffae293 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:52.275343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b04adbcb-8e25-41f9-9364-88700ffae293 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:52.275820 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b04adbcb-8e25-41f9-9364-88700ffae293 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:52.276288 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b04adbcb-8e25-41f9-9364-88700ffae293 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:52.276975 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b04adbcb-8e25-41f9-9364-88700ffae293 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:52.277421 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b04adbcb-8e25-41f9-9364-88700ffae293 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:52.277839 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b04adbcb-8e25-41f9-9364-88700ffae293 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:52.308457 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b04adbcb-8e25-41f9-9364-88700ffae293 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047538 Oct 24 08:58:52.308935 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 376/747] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 08:58:52 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:58:52.579271 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b5a4e66d-aba5-424c-9543-723cff9c87e4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:52.579870 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b5a4e66d-aba5-424c-9543-723cff9c87e4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:52.586515 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b5a4e66d-aba5-424c-9543-723cff9c87e4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:52.587020 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b5a4e66d-aba5-424c-9543-723cff9c87e4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:52.587465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b5a4e66d-aba5-424c-9543-723cff9c87e4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:52.588218 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b5a4e66d-aba5-424c-9543-723cff9c87e4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:52.588682 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b5a4e66d-aba5-424c-9543-723cff9c87e4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:52.589110 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b5a4e66d-aba5-424c-9543-723cff9c87e4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:52.728492 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b5a4e66d-aba5-424c-9543-723cff9c87e4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:52.736830 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b5a4e66d-aba5-424c-9543-723cff9c87e4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:52.738285 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b5a4e66d-aba5-424c-9543-723cff9c87e4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:52.739703 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b5a4e66d-aba5-424c-9543-723cff9c87e4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1723 microversion: 2.20 time: 0.165058 Oct 24 08:58:52.740284 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 372/748] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:58:52 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1723 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:58:53.322299 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-dab6d240-88ba-4a1c-84ff-fcd27fc261a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:53.324527 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-dab6d240-88ba-4a1c-84ff-fcd27fc261a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:53.331147 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dab6d240-88ba-4a1c-84ff-fcd27fc261a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:53.331626 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dab6d240-88ba-4a1c-84ff-fcd27fc261a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:53.332063 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dab6d240-88ba-4a1c-84ff-fcd27fc261a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:53.332784 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dab6d240-88ba-4a1c-84ff-fcd27fc261a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:53.333234 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dab6d240-88ba-4a1c-84ff-fcd27fc261a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:53.333656 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dab6d240-88ba-4a1c-84ff-fcd27fc261a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:53.362611 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-dab6d240-88ba-4a1c-84ff-fcd27fc261a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044279 Oct 24 08:58:53.363171 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 377/749] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 08:58:53 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 24 08:58:53.427539 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2059ce40-c215-41d6-843f-af3744d9e856 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:53.428030 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2059ce40-c215-41d6-843f-af3744d9e856 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:53.434074 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2059ce40-c215-41d6-843f-af3744d9e856 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:53.434561 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2059ce40-c215-41d6-843f-af3744d9e856 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:53.435001 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2059ce40-c215-41d6-843f-af3744d9e856 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:53.435701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2059ce40-c215-41d6-843f-af3744d9e856 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:53.436180 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2059ce40-c215-41d6-843f-af3744d9e856 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:53.436612 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2059ce40-c215-41d6-843f-af3744d9e856 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:53.461710 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2059ce40-c215-41d6-843f-af3744d9e856 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Going to try to terminate instance {{(pid=130396) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 08:58:53.523115 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=93,nova_api:DELETE=1,nova_api:INSERT=6 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:58:53.541722 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2059ce40-c215-41d6-843f-af3744d9e856 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "DELETE /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 204 len: 0 microversion: 2.1 time: 0.117397 Oct 24 08:58:53.546068 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 373/750] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 08:58:53 2022] DELETE /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 24 08:58:53.552309 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-2b9a221b-e505-44e7-a1af-c2a0563f92aa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:53.552922 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-2b9a221b-e505-44e7-a1af-c2a0563f92aa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:53.559122 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2b9a221b-e505-44e7-a1af-c2a0563f92aa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:53.559608 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2b9a221b-e505-44e7-a1af-c2a0563f92aa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:53.560107 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2b9a221b-e505-44e7-a1af-c2a0563f92aa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:53.560880 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2b9a221b-e505-44e7-a1af-c2a0563f92aa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:53.561334 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2b9a221b-e505-44e7-a1af-c2a0563f92aa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:53.561756 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2b9a221b-e505-44e7-a1af-c2a0563f92aa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:53.693292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-2b9a221b-e505-44e7-a1af-c2a0563f92aa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:53.701060 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-2b9a221b-e505-44e7-a1af-c2a0563f92aa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 200 len: 1683 microversion: 2.1 time: 0.153079 Oct 24 08:58:53.701628 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 378/751] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:53 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 1683 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:53.766635 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ab811935-463d-4386-a6da-8de0e390dbd5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:53.767330 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ab811935-463d-4386-a6da-8de0e390dbd5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:53.774400 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ab811935-463d-4386-a6da-8de0e390dbd5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:53.774886 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ab811935-463d-4386-a6da-8de0e390dbd5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:53.775320 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ab811935-463d-4386-a6da-8de0e390dbd5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:53.776014 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ab811935-463d-4386-a6da-8de0e390dbd5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:53.776562 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ab811935-463d-4386-a6da-8de0e390dbd5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:53.776987 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ab811935-463d-4386-a6da-8de0e390dbd5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:53.913029 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ab811935-463d-4386-a6da-8de0e390dbd5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:53.921122 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ab811935-463d-4386-a6da-8de0e390dbd5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:53.923237 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ab811935-463d-4386-a6da-8de0e390dbd5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:53.924427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ab811935-463d-4386-a6da-8de0e390dbd5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1723 microversion: 2.20 time: 0.162405 Oct 24 08:58:53.924993 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 374/752] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:58:53 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1723 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:58:54.724802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b41fe615-1518-4697-85ef-38e4ad459eaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:54.725413 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b41fe615-1518-4697-85ef-38e4ad459eaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:54.731793 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b41fe615-1518-4697-85ef-38e4ad459eaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:54.732413 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b41fe615-1518-4697-85ef-38e4ad459eaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:54.732919 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b41fe615-1518-4697-85ef-38e4ad459eaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:54.733558 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b41fe615-1518-4697-85ef-38e4ad459eaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:54.734548 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b41fe615-1518-4697-85ef-38e4ad459eaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:54.737481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b41fe615-1518-4697-85ef-38e4ad459eaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:54.896829 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-b41fe615-1518-4697-85ef-38e4ad459eaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:54.907374 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b41fe615-1518-4697-85ef-38e4ad459eaa tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 200 len: 1683 microversion: 2.1 time: 0.186345 Oct 24 08:58:54.907960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 379/753] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:54 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 1683 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:54.949911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-253ba0ce-b52e-4bd6-a13e-fa7d16fdfc14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:54.950521 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-253ba0ce-b52e-4bd6-a13e-fa7d16fdfc14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:54.957267 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-253ba0ce-b52e-4bd6-a13e-fa7d16fdfc14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:54.957759 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-253ba0ce-b52e-4bd6-a13e-fa7d16fdfc14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:54.958195 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-253ba0ce-b52e-4bd6-a13e-fa7d16fdfc14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:54.958979 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-253ba0ce-b52e-4bd6-a13e-fa7d16fdfc14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:54.959434 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-253ba0ce-b52e-4bd6-a13e-fa7d16fdfc14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:54.959886 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-253ba0ce-b52e-4bd6-a13e-fa7d16fdfc14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:55.100391 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-253ba0ce-b52e-4bd6-a13e-fa7d16fdfc14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:55.108400 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-253ba0ce-b52e-4bd6-a13e-fa7d16fdfc14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:55.110682 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-253ba0ce-b52e-4bd6-a13e-fa7d16fdfc14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:55.111860 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-253ba0ce-b52e-4bd6-a13e-fa7d16fdfc14 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1723 microversion: 2.20 time: 0.165936 Oct 24 08:58:55.112453 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 375/754] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:58:54 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1723 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:58:55.934147 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-dedd5fc1-b949-4805-a68b-0f593aed6eea tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:55.935399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-dedd5fc1-b949-4805-a68b-0f593aed6eea tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:55.938956 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=77,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:58:55.944977 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dedd5fc1-b949-4805-a68b-0f593aed6eea tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:55.945459 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dedd5fc1-b949-4805-a68b-0f593aed6eea tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:55.945897 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dedd5fc1-b949-4805-a68b-0f593aed6eea tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:55.946610 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dedd5fc1-b949-4805-a68b-0f593aed6eea tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:55.947126 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dedd5fc1-b949-4805-a68b-0f593aed6eea tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:55.947558 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dedd5fc1-b949-4805-a68b-0f593aed6eea tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:56.137128 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-142301df-1f90-43ee-ba12-cc788a27fcba tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:56.137739 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-142301df-1f90-43ee-ba12-cc788a27fcba tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:56.143939 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-142301df-1f90-43ee-ba12-cc788a27fcba tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:56.152963 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-142301df-1f90-43ee-ba12-cc788a27fcba tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:56.153509 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-142301df-1f90-43ee-ba12-cc788a27fcba tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:56.154267 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-142301df-1f90-43ee-ba12-cc788a27fcba tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:56.154749 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-142301df-1f90-43ee-ba12-cc788a27fcba tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:56.155202 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-142301df-1f90-43ee-ba12-cc788a27fcba tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:56.164444 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-dedd5fc1-b949-4805-a68b-0f593aed6eea tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:56.178572 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-dedd5fc1-b949-4805-a68b-0f593aed6eea tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 200 len: 1683 microversion: 2.1 time: 0.246222 Oct 24 08:58:56.188026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 380/755] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:55 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 1683 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:56.410963 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-142301df-1f90-43ee-ba12-cc788a27fcba tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:56.419844 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-142301df-1f90-43ee-ba12-cc788a27fcba tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:56.421371 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-142301df-1f90-43ee-ba12-cc788a27fcba tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:56.423093 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-142301df-1f90-43ee-ba12-cc788a27fcba tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1723 microversion: 2.20 time: 0.289954 Oct 24 08:58:56.423945 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 376/756] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:58:56 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1723 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:58:57.131407 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-e65df35b-2bc3-473f-bf21-30f6a2d592e4 req-38e65a62-c4a7-41ad-80da-21f0f264aaac service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf", "name": "network-vif-deleted", "tag": "9d796584-37f8-4f4f-8f8c-41d40022f76b"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:58:57.143659 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-e65df35b-2bc3-473f-bf21-30f6a2d592e4 req-38e65a62-c4a7-41ad-80da-21f0f264aaac service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:57.144172 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-e65df35b-2bc3-473f-bf21-30f6a2d592e4 req-38e65a62-c4a7-41ad-80da-21f0f264aaac service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:57.144590 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-e65df35b-2bc3-473f-bf21-30f6a2d592e4 req-38e65a62-c4a7-41ad-80da-21f0f264aaac service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:57.168531 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-e65df35b-2bc3-473f-bf21-30f6a2d592e4 req-38e65a62-c4a7-41ad-80da-21f0f264aaac service nova] Creating event network-vif-deleted:9d796584-37f8-4f4f-8f8c-41d40022f76b for instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf on hv-cinder-maste Oct 24 08:58:57.173415 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-e65df35b-2bc3-473f-bf21-30f6a2d592e4 req-38e65a62-c4a7-41ad-80da-21f0f264aaac service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045892 Oct 24 08:58:57.174019 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 381/757] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:58:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:58:57.204322 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-cd8a6362-f821-489b-bb35-40909527e190 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:57.205507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-cd8a6362-f821-489b-bb35-40909527e190 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:57.211978 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cd8a6362-f821-489b-bb35-40909527e190 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:57.212494 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cd8a6362-f821-489b-bb35-40909527e190 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:57.212945 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cd8a6362-f821-489b-bb35-40909527e190 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:57.213665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cd8a6362-f821-489b-bb35-40909527e190 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:57.214096 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cd8a6362-f821-489b-bb35-40909527e190 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:57.214515 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cd8a6362-f821-489b-bb35-40909527e190 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:57.304289 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-cd8a6362-f821-489b-bb35-40909527e190 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:57.315323 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-cd8a6362-f821-489b-bb35-40909527e190 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 200 len: 1327 microversion: 2.1 time: 0.115195 Oct 24 08:58:57.328173 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 377/758] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:57 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 1327 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:58:57.450073 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8cd7826c-f5ec-4839-a3d7-4a61d87d5c04 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:57.451035 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8cd7826c-f5ec-4839-a3d7-4a61d87d5c04 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:57.458477 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8cd7826c-f5ec-4839-a3d7-4a61d87d5c04 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:57.459064 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8cd7826c-f5ec-4839-a3d7-4a61d87d5c04 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:57.459742 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8cd7826c-f5ec-4839-a3d7-4a61d87d5c04 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:57.460715 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8cd7826c-f5ec-4839-a3d7-4a61d87d5c04 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:57.461408 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8cd7826c-f5ec-4839-a3d7-4a61d87d5c04 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:57.462093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8cd7826c-f5ec-4839-a3d7-4a61d87d5c04 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:57.650270 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8cd7826c-f5ec-4839-a3d7-4a61d87d5c04 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:57.659512 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8cd7826c-f5ec-4839-a3d7-4a61d87d5c04 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:57.661019 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8cd7826c-f5ec-4839-a3d7-4a61d87d5c04 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:57.662224 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8cd7826c-f5ec-4839-a3d7-4a61d87d5c04 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1723 microversion: 2.20 time: 0.216452 Oct 24 08:58:57.662778 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 382/759] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:58:57 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1723 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:58:58.340091 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-588834dc-98e4-4052-a8dd-a39c976e0650 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:58.340720 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-588834dc-98e4-4052-a8dd-a39c976e0650 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:58.347693 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-588834dc-98e4-4052-a8dd-a39c976e0650 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:58.348209 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-588834dc-98e4-4052-a8dd-a39c976e0650 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:58.348654 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-588834dc-98e4-4052-a8dd-a39c976e0650 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:58.349485 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-588834dc-98e4-4052-a8dd-a39c976e0650 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:58.349822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-588834dc-98e4-4052-a8dd-a39c976e0650 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:58.350243 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-588834dc-98e4-4052-a8dd-a39c976e0650 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:58.364625 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-588834dc-98e4-4052-a8dd-a39c976e0650 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] HTTP exception thrown: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-588834dc-98e4-4052-a8dd-a39c976e0650 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Request method failure captured: Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf HTTP/1.1 Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlH_P2cgqdGxTyt_DjhRKpesrNF98h-yweC7dRlZggOYZXXonk3ROJmezNObJqlbezl2V7Pqsl8fDKOVcqEGBVvIZd3l8-ZBbHliak_eivuyadmAWxfis-R_etsiC3KVVo1LvGlKxFu0GFkQBrTFC5gIfs64iGB92PfDe2tNtbCL85U Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 08:58:58.365699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 08:58:58.367319 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b34f10ec378a4622a8f0b0546e84da7f", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}] Oct 24 08:58:58.367319 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-AttachVolumeNegativeTest-97597731 Oct 24 08:58:58.367319 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 08:58:58.367319 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 08:58:58.367319 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 08:58:58.367319 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 08:58:58.367319 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 08:58:58.367319 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: ebdef1d69bb14837b9a0c28cbba332fc Oct 24 08:58:58.367319 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 08:58:58.367319 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 08:58:58.367319 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. Oct 24 08:58:58.367319 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': 'd9301ca5-bdaa-4f5d-8bcd-772b8923b1cf'} Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:58:58.368295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. Oct 24 08:58:58.369863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:58:58.371548 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-588834dc-98e4-4052-a8dd-a39c976e0650 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Returning 404 to user: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:58:58.371548 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-588834dc-98e4-4052-a8dd-a39c976e0650 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 404 len: 111 microversion: 2.1 time: 0.030839 Oct 24 08:58:58.371548 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 378/760] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:58:58 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 08:58:58.376895 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a7a9fdf2-564a-485a-b456-298a1fdfe5db tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '.version_select at 0x7fdfedd20940>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:58.398593 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a7a9fdf2-564a-485a-b456-298a1fdfe5db tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1846818907" status: 202 len: 0 microversion: 2.1 time: 0.026199 Oct 24 08:58:58.399447 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 383/761] 10.222.0.39 () {58 vars in 1221 bytes} [Mon Oct 24 08:58:58 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1846818907 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 08:58:58.688693 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b0f22d52-fffe-41d7-a9a4-90b420a134bc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:58.689451 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b0f22d52-fffe-41d7-a9a4-90b420a134bc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:58.695632 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0f22d52-fffe-41d7-a9a4-90b420a134bc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:58.696210 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0f22d52-fffe-41d7-a9a4-90b420a134bc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:58.696701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0f22d52-fffe-41d7-a9a4-90b420a134bc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:58.697578 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0f22d52-fffe-41d7-a9a4-90b420a134bc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:58.698292 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0f22d52-fffe-41d7-a9a4-90b420a134bc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:58.698766 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b0f22d52-fffe-41d7-a9a4-90b420a134bc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:58.703505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:58:58.923701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b0f22d52-fffe-41d7-a9a4-90b420a134bc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:58.933014 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b0f22d52-fffe-41d7-a9a4-90b420a134bc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:58.934461 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b0f22d52-fffe-41d7-a9a4-90b420a134bc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:58:58.935627 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b0f22d52-fffe-41d7-a9a4-90b420a134bc tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1723 microversion: 2.20 time: 0.251618 Oct 24 08:58:58.936206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 379/762] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:58:58 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1723 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:58:59.195364 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ae1dafcf-faa4-4347-9bca-97911d9a96d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Action: 'create', calling method: .version_select at 0x7fdfedc68940>, body: {"keypair": {"name": "tempest-keypair-186264620", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAX53URGZBgBjxIrujGVmyN0Ne2rb1aAcMFVtq52C6tcqtTiXLNSVyF9c5//gkgK2rYefob/5B4qBcT5I5kMdVnrhJ4smSxlZKv+YTZe/2u5sCP9zmtpOjs8UFdXtPFelw=="}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:58:59.213283 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-ae1dafcf-faa4-4347-9bca-97911d9a96d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Getting quotas for project b34f10ec378a4622a8f0b0546e84da7f. Resources: {'key_pairs'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:58:59.217185 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-ae1dafcf-faa4-4347-9bca-97911d9a96d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Getting quotas for user ebdef1d69bb14837b9a0c28cbba332fc and project b34f10ec378a4622a8f0b0546e84da7f. Resources: {'key_pairs'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:58:59.240004 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ae1dafcf-faa4-4347-9bca-97911d9a96d0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.048462 Oct 24 08:58:59.240638 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 384/763] 10.222.0.39 () {60 vars in 1159 bytes} [Mon Oct 24 08:58:59 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:58:59.961578 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-165e542b-1b38-4f87-b89a-5bf7c1082f49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:58:59.962450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-165e542b-1b38-4f87-b89a-5bf7c1082f49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:58:59.970526 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-165e542b-1b38-4f87-b89a-5bf7c1082f49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:59.971016 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-165e542b-1b38-4f87-b89a-5bf7c1082f49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:59.971644 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-165e542b-1b38-4f87-b89a-5bf7c1082f49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:58:59.972715 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-165e542b-1b38-4f87-b89a-5bf7c1082f49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:58:59.972971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-165e542b-1b38-4f87-b89a-5bf7c1082f49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:58:59.973395 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-165e542b-1b38-4f87-b89a-5bf7c1082f49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:00.120709 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-165e542b-1b38-4f87-b89a-5bf7c1082f49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:00.129383 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-165e542b-1b38-4f87-b89a-5bf7c1082f49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:00.130907 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-165e542b-1b38-4f87-b89a-5bf7c1082f49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:00.132198 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-165e542b-1b38-4f87-b89a-5bf7c1082f49 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1723 microversion: 2.20 time: 0.174601 Oct 24 08:59:00.132851 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 380/764] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:58:59 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1723 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:00.565263 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-347593903", "imageRef": "bba331ba-faf5-4a56-81d9-6e84e9609665", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "151b561f-53f8-4019-8ed3-b08a9e67fe57"}], "security_groups": [{"name": "tempest-securitygroup--63723430"}], "key_name": "tempest-keypair-186264620", "user_data": "}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:59:00.671059 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.network.neutron [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] validate_networks() for [('151b561f-53f8-4019-8ed3-b08a9e67fe57', None, None, None, None, None)] {{(pid=130395) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 08:59:01.049512 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Flavor limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 08:59:01.049938 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Image limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 08:59:01.050415 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Flavor pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 08:59:01.050826 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Image pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 08:59:01.051238 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 08:59:01.060073 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:01.060594 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:01.061035 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:01.061765 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:01.062240 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:01.062890 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:01.079627 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Getting quotas for project b34f10ec378a4622a8f0b0546e84da7f. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 08:59:01.083682 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Getting quotas for user ebdef1d69bb14837b9a0c28cbba332fc and project b34f10ec378a4622a8f0b0546e84da7f. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 08:59:01.096802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Going to run 1 instances... {{(pid=130395) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 08:59:01.099686 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bba331ba-faf5-4a56-81d9-6e84e9609665',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130395) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 08:59:01.149963 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d60ed9bf-2f20-449e-8788-200e5aee1940 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.1 time: 0.590613 Oct 24 08:59:01.150616 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 385/765] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 08:59:00 2022] POST /compute/v2.1/servers => generated 400 bytes in 592 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 08:59:01.159479 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f4b0eb31-cd44-4a52-9a4c-9f79b1e1debe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:01.160695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f4b0eb31-cd44-4a52-9a4c-9f79b1e1debe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:01.161716 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-67bf0ec3-629c-4594-877b-12f1b9bd1cf7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:01.162444 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-67bf0ec3-629c-4594-877b-12f1b9bd1cf7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:01.167421 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f4b0eb31-cd44-4a52-9a4c-9f79b1e1debe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:01.167909 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f4b0eb31-cd44-4a52-9a4c-9f79b1e1debe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:01.168382 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f4b0eb31-cd44-4a52-9a4c-9f79b1e1debe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:01.169275 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f4b0eb31-cd44-4a52-9a4c-9f79b1e1debe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:01.170112 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f4b0eb31-cd44-4a52-9a4c-9f79b1e1debe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:01.170545 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f4b0eb31-cd44-4a52-9a4c-9f79b1e1debe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:01.221873 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-67bf0ec3-629c-4594-877b-12f1b9bd1cf7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:01.230124 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-67bf0ec3-629c-4594-877b-12f1b9bd1cf7 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1240 microversion: 2.1 time: 0.072239 Oct 24 08:59:01.231023 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 386/766] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:01 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1240 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:59:01.337809 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f4b0eb31-cd44-4a52-9a4c-9f79b1e1debe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:01.346487 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f4b0eb31-cd44-4a52-9a4c-9f79b1e1debe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:01.347932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f4b0eb31-cd44-4a52-9a4c-9f79b1e1debe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:01.349126 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f4b0eb31-cd44-4a52-9a4c-9f79b1e1debe tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1723 microversion: 2.20 time: 0.194104 Oct 24 08:59:01.354708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 381/767] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:01 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1723 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:59:02.253724 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b6cb0f24-a988-45ac-ba96-7afe43b80cad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:02.254685 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b6cb0f24-a988-45ac-ba96-7afe43b80cad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:02.261201 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6cb0f24-a988-45ac-ba96-7afe43b80cad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:02.261691 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6cb0f24-a988-45ac-ba96-7afe43b80cad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:02.262139 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6cb0f24-a988-45ac-ba96-7afe43b80cad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:02.262910 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6cb0f24-a988-45ac-ba96-7afe43b80cad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:02.263368 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6cb0f24-a988-45ac-ba96-7afe43b80cad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:02.263795 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6cb0f24-a988-45ac-ba96-7afe43b80cad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:02.331019 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-b6cb0f24-a988-45ac-ba96-7afe43b80cad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:02.339059 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b6cb0f24-a988-45ac-ba96-7afe43b80cad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1292 microversion: 2.1 time: 0.089724 Oct 24 08:59:02.339625 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 387/768] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:02 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1292 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:02.376758 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ff758ff5-a8de-4c44-85e4-0ded93628fcd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:02.377461 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ff758ff5-a8de-4c44-85e4-0ded93628fcd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:02.383632 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff758ff5-a8de-4c44-85e4-0ded93628fcd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:02.384115 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff758ff5-a8de-4c44-85e4-0ded93628fcd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:02.384785 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff758ff5-a8de-4c44-85e4-0ded93628fcd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:02.385630 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff758ff5-a8de-4c44-85e4-0ded93628fcd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:02.386086 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff758ff5-a8de-4c44-85e4-0ded93628fcd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:02.386639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff758ff5-a8de-4c44-85e4-0ded93628fcd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:02.517490 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ff758ff5-a8de-4c44-85e4-0ded93628fcd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:02.525498 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ff758ff5-a8de-4c44-85e4-0ded93628fcd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:02.526958 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ff758ff5-a8de-4c44-85e4-0ded93628fcd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:02.528099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ff758ff5-a8de-4c44-85e4-0ded93628fcd tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1723 microversion: 2.20 time: 0.157003 Oct 24 08:59:02.528677 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 382/769] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:02 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1723 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:03.364101 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7fd40a9c-4987-4fc6-887e-fa6c8a3e1c46 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:03.364746 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7fd40a9c-4987-4fc6-887e-fa6c8a3e1c46 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:03.371085 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7fd40a9c-4987-4fc6-887e-fa6c8a3e1c46 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:03.371568 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7fd40a9c-4987-4fc6-887e-fa6c8a3e1c46 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:03.372006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7fd40a9c-4987-4fc6-887e-fa6c8a3e1c46 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:03.372728 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7fd40a9c-4987-4fc6-887e-fa6c8a3e1c46 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:03.373179 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7fd40a9c-4987-4fc6-887e-fa6c8a3e1c46 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:03.373602 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7fd40a9c-4987-4fc6-887e-fa6c8a3e1c46 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:03.394053 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:03.441372 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7fd40a9c-4987-4fc6-887e-fa6c8a3e1c46 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:03.449250 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7fd40a9c-4987-4fc6-887e-fa6c8a3e1c46 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1298 microversion: 2.1 time: 0.089252 Oct 24 08:59:03.449821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 388/770] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:03 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1298 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:03.555593 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f9eb79ad-aa31-4943-ade5-eb70a89c73b5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:03.556198 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f9eb79ad-aa31-4943-ade5-eb70a89c73b5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:03.563847 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f9eb79ad-aa31-4943-ade5-eb70a89c73b5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:03.564418 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f9eb79ad-aa31-4943-ade5-eb70a89c73b5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:03.564884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f9eb79ad-aa31-4943-ade5-eb70a89c73b5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:03.565656 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f9eb79ad-aa31-4943-ade5-eb70a89c73b5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:03.566121 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f9eb79ad-aa31-4943-ade5-eb70a89c73b5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:03.566550 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f9eb79ad-aa31-4943-ade5-eb70a89c73b5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:03.706324 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f9eb79ad-aa31-4943-ade5-eb70a89c73b5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:03.714344 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f9eb79ad-aa31-4943-ade5-eb70a89c73b5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:03.715773 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f9eb79ad-aa31-4943-ade5-eb70a89c73b5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:03.716943 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f9eb79ad-aa31-4943-ade5-eb70a89c73b5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1723 microversion: 2.20 time: 0.165418 Oct 24 08:59:03.717499 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 383/771] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:03 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1723 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:04.474220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-2f630851-cbd0-4a9f-8d9c-8d6596de1885 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:04.474837 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-2f630851-cbd0-4a9f-8d9c-8d6596de1885 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:04.480855 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2f630851-cbd0-4a9f-8d9c-8d6596de1885 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:04.481418 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2f630851-cbd0-4a9f-8d9c-8d6596de1885 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:04.481922 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2f630851-cbd0-4a9f-8d9c-8d6596de1885 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:04.482692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2f630851-cbd0-4a9f-8d9c-8d6596de1885 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:04.483219 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2f630851-cbd0-4a9f-8d9c-8d6596de1885 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:04.483653 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2f630851-cbd0-4a9f-8d9c-8d6596de1885 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:04.565072 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-2f630851-cbd0-4a9f-8d9c-8d6596de1885 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:04.572624 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-2f630851-cbd0-4a9f-8d9c-8d6596de1885 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1298 microversion: 2.1 time: 0.102568 Oct 24 08:59:04.573198 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 389/772] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:04 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1298 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:04.743507 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2f6cbb52-1e6c-435f-bbbf-1656e62ae6f5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:04.744908 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2f6cbb52-1e6c-435f-bbbf-1656e62ae6f5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:04.753722 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2f6cbb52-1e6c-435f-bbbf-1656e62ae6f5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:04.754705 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2f6cbb52-1e6c-435f-bbbf-1656e62ae6f5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:04.755807 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2f6cbb52-1e6c-435f-bbbf-1656e62ae6f5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:04.756607 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2f6cbb52-1e6c-435f-bbbf-1656e62ae6f5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:04.757367 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2f6cbb52-1e6c-435f-bbbf-1656e62ae6f5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:04.757995 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2f6cbb52-1e6c-435f-bbbf-1656e62ae6f5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:04.978490 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2f6cbb52-1e6c-435f-bbbf-1656e62ae6f5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:04.991321 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2f6cbb52-1e6c-435f-bbbf-1656e62ae6f5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:04.993165 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2f6cbb52-1e6c-435f-bbbf-1656e62ae6f5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:04.995219 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2f6cbb52-1e6c-435f-bbbf-1656e62ae6f5 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1744 microversion: 2.20 time: 0.255756 Oct 24 08:59:04.995560 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 384/773] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:04 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1744 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:05.513396 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-8cd1d011-0c02-4f46-952a-b147531195dc req-f16847b0-e8aa-4793-a224-2333eadef0fa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "240dec37-c7ff-4c59-8b5f-611a52575853", "tag": "86414159-5cb9-488f-8380-698e8b8e4c32"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:59:05.524615 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-8cd1d011-0c02-4f46-952a-b147531195dc req-f16847b0-e8aa-4793-a224-2333eadef0fa service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:05.525100 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-8cd1d011-0c02-4f46-952a-b147531195dc req-f16847b0-e8aa-4793-a224-2333eadef0fa service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:05.525537 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-8cd1d011-0c02-4f46-952a-b147531195dc req-f16847b0-e8aa-4793-a224-2333eadef0fa service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:05.538602 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-8cd1d011-0c02-4f46-952a-b147531195dc req-f16847b0-e8aa-4793-a224-2333eadef0fa service nova] Creating event network-changed:86414159-5cb9-488f-8380-698e8b8e4c32 for instance 240dec37-c7ff-4c59-8b5f-611a52575853 on hv-cinder-maste Oct 24 08:59:05.543404 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-8cd1d011-0c02-4f46-952a-b147531195dc req-f16847b0-e8aa-4793-a224-2333eadef0fa service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034434 Oct 24 08:59:05.544012 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 390/774] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:59:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:59:05.595600 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b61dbac5-8083-4bee-8295-f13f2e17e8a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:05.596148 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b61dbac5-8083-4bee-8295-f13f2e17e8a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:05.602286 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b61dbac5-8083-4bee-8295-f13f2e17e8a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:05.602772 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b61dbac5-8083-4bee-8295-f13f2e17e8a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:05.603203 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b61dbac5-8083-4bee-8295-f13f2e17e8a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:05.604603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b61dbac5-8083-4bee-8295-f13f2e17e8a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:05.605081 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b61dbac5-8083-4bee-8295-f13f2e17e8a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:05.605747 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b61dbac5-8083-4bee-8295-f13f2e17e8a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:05.739967 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b61dbac5-8083-4bee-8295-f13f2e17e8a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:05.749401 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b61dbac5-8083-4bee-8295-f13f2e17e8a1 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1364 microversion: 2.1 time: 0.158155 Oct 24 08:59:05.756578 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 385/775] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:05 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1364 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:06.022343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-68854b4e-eaab-4301-a9ab-285c3289dacb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:06.023164 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-68854b4e-eaab-4301-a9ab-285c3289dacb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:06.029581 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-68854b4e-eaab-4301-a9ab-285c3289dacb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:06.030362 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-68854b4e-eaab-4301-a9ab-285c3289dacb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:06.031603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-68854b4e-eaab-4301-a9ab-285c3289dacb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:06.032479 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-68854b4e-eaab-4301-a9ab-285c3289dacb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:06.033240 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-68854b4e-eaab-4301-a9ab-285c3289dacb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:06.033935 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-68854b4e-eaab-4301-a9ab-285c3289dacb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:06.224697 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-68854b4e-eaab-4301-a9ab-285c3289dacb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:06.233008 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-68854b4e-eaab-4301-a9ab-285c3289dacb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:06.234758 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-68854b4e-eaab-4301-a9ab-285c3289dacb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:06.236400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-68854b4e-eaab-4301-a9ab-285c3289dacb tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1744 microversion: 2.20 time: 0.218269 Oct 24 08:59:06.237048 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 391/776] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:06 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1744 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:06.772521 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-748a2541-9f88-43e3-a7d2-8ea2614b199a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:06.773175 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-748a2541-9f88-43e3-a7d2-8ea2614b199a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:06.779525 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-748a2541-9f88-43e3-a7d2-8ea2614b199a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:06.780009 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-748a2541-9f88-43e3-a7d2-8ea2614b199a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:06.781157 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-748a2541-9f88-43e3-a7d2-8ea2614b199a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:06.781874 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-748a2541-9f88-43e3-a7d2-8ea2614b199a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:06.782342 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-748a2541-9f88-43e3-a7d2-8ea2614b199a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:06.782765 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-748a2541-9f88-43e3-a7d2-8ea2614b199a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:06.917540 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-748a2541-9f88-43e3-a7d2-8ea2614b199a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:06.925601 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-748a2541-9f88-43e3-a7d2-8ea2614b199a tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1364 microversion: 2.1 time: 0.156909 Oct 24 08:59:06.926169 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 386/777] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:06 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1364 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:07.263265 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-0fbfd781-0373-43b5-a6b0-8724ae076b9c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:07.263868 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-0fbfd781-0373-43b5-a6b0-8724ae076b9c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:07.270293 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0fbfd781-0373-43b5-a6b0-8724ae076b9c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:07.270771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0fbfd781-0373-43b5-a6b0-8724ae076b9c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:07.271207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0fbfd781-0373-43b5-a6b0-8724ae076b9c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:07.271902 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0fbfd781-0373-43b5-a6b0-8724ae076b9c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:07.272390 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0fbfd781-0373-43b5-a6b0-8724ae076b9c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:07.272814 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0fbfd781-0373-43b5-a6b0-8724ae076b9c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:07.405503 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0fbfd781-0373-43b5-a6b0-8724ae076b9c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:07.413671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0fbfd781-0373-43b5-a6b0-8724ae076b9c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:07.415118 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0fbfd781-0373-43b5-a6b0-8724ae076b9c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:07.416305 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-0fbfd781-0373-43b5-a6b0-8724ae076b9c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1744 microversion: 2.20 time: 0.156880 Oct 24 08:59:07.416868 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 392/778] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:07 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1744 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:07.781395 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-4ac58900-8221-406e-bcde-af0feaeb6930 req-5af87b7f-8234-476a-ab06-2649700667fa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e72acc7a-decb-46c7-9c51-f731e2c413b3", "name": "network-vif-plugged", "status": "completed", "tag": "d0e79b5c-b764-405d-8c7a-d5032e3c72a8"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:59:07.792608 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-4ac58900-8221-406e-bcde-af0feaeb6930 req-5af87b7f-8234-476a-ab06-2649700667fa service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:07.793636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-4ac58900-8221-406e-bcde-af0feaeb6930 req-5af87b7f-8234-476a-ab06-2649700667fa service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:07.794101 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-4ac58900-8221-406e-bcde-af0feaeb6930 req-5af87b7f-8234-476a-ab06-2649700667fa service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:07.806597 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-4ac58900-8221-406e-bcde-af0feaeb6930 req-5af87b7f-8234-476a-ab06-2649700667fa service nova] Creating event network-vif-plugged:d0e79b5c-b764-405d-8c7a-d5032e3c72a8 for instance e72acc7a-decb-46c7-9c51-f731e2c413b3 on hv-cinder-maste Oct 24 08:59:07.811975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-4ac58900-8221-406e-bcde-af0feaeb6930 req-5af87b7f-8234-476a-ab06-2649700667fa service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034714 Oct 24 08:59:07.812727 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 387/779] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:59:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:59:07.958094 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-73d32ed6-6e09-42a1-9398-3ac551ea47be tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:07.958799 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-73d32ed6-6e09-42a1-9398-3ac551ea47be tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:07.964982 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-73d32ed6-6e09-42a1-9398-3ac551ea47be tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:07.965473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-73d32ed6-6e09-42a1-9398-3ac551ea47be tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:07.965913 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-73d32ed6-6e09-42a1-9398-3ac551ea47be tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:07.966621 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-73d32ed6-6e09-42a1-9398-3ac551ea47be tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:07.967077 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-73d32ed6-6e09-42a1-9398-3ac551ea47be tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:07.967500 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-73d32ed6-6e09-42a1-9398-3ac551ea47be tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:08.004803 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5b6f287b-c27d-4e3c-abf1-4debd01a5e39 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e8c3dfe6-2fa6-475f-8efe-2cf4bc2b8fa4", "device": "/dev/sdb"}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:59:08.007745 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-5b6f287b-c27d-4e3c-abf1-4debd01a5e39 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:08.015987 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5b6f287b-c27d-4e3c-abf1-4debd01a5e39 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:08.016510 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5b6f287b-c27d-4e3c-abf1-4debd01a5e39 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:08.016946 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5b6f287b-c27d-4e3c-abf1-4debd01a5e39 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:08.017645 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5b6f287b-c27d-4e3c-abf1-4debd01a5e39 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:08.018091 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5b6f287b-c27d-4e3c-abf1-4debd01a5e39 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:08.018508 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5b6f287b-c27d-4e3c-abf1-4debd01a5e39 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:08.103273 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-73d32ed6-6e09-42a1-9398-3ac551ea47be tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:08.111561 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-73d32ed6-6e09-42a1-9398-3ac551ea47be tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1364 microversion: 2.1 time: 0.157423 Oct 24 08:59:08.112162 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 393/780] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:07 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1364 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:08.443328 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-4d7150a6-367f-483b-936f-575207b76342 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:08.443976 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-4d7150a6-367f-483b-936f-575207b76342 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:08.450348 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d7150a6-367f-483b-936f-575207b76342 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:08.450826 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d7150a6-367f-483b-936f-575207b76342 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:08.451265 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d7150a6-367f-483b-936f-575207b76342 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:08.451961 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d7150a6-367f-483b-936f-575207b76342 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:08.452454 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d7150a6-367f-483b-936f-575207b76342 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:08.453118 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4d7150a6-367f-483b-936f-575207b76342 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:08.581457 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-4d7150a6-367f-483b-936f-575207b76342 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:08.589551 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-4d7150a6-367f-483b-936f-575207b76342 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:08.590994 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-4d7150a6-367f-483b-936f-575207b76342 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:08.592175 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-4d7150a6-367f-483b-936f-575207b76342 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1744 microversion: 2.20 time: 0.153198 Oct 24 08:59:08.592745 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 394/781] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:08 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1744 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:59:09.134858 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1c9187f9-8b23-41cf-829f-1d497382e4ad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:09.135470 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1c9187f9-8b23-41cf-829f-1d497382e4ad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:09.141971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1c9187f9-8b23-41cf-829f-1d497382e4ad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:09.142463 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1c9187f9-8b23-41cf-829f-1d497382e4ad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:09.142907 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1c9187f9-8b23-41cf-829f-1d497382e4ad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:09.143657 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1c9187f9-8b23-41cf-829f-1d497382e4ad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:09.144115 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1c9187f9-8b23-41cf-829f-1d497382e4ad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:09.144566 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1c9187f9-8b23-41cf-829f-1d497382e4ad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:09.372816 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1c9187f9-8b23-41cf-829f-1d497382e4ad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:09.378651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5b6f287b-c27d-4e3c-abf1-4debd01a5e39 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "POST /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.378543 Oct 24 08:59:09.379586 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 388/782] 10.222.0.39 () {60 vars in 1323 bytes} [Mon Oct 24 08:59:07 2022] POST /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181/os-volume_attachments => generated 194 bytes in 1381 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:59:09.382345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1c9187f9-8b23-41cf-829f-1d497382e4ad tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1364 microversion: 2.1 time: 0.251238 Oct 24 08:59:09.383112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 395/783] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:09 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1364 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:59:09.619912 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-1d24e8ea-9446-49e9-8e86-deec6992c9d6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:09.620635 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-1d24e8ea-9446-49e9-8e86-deec6992c9d6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:09.626903 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1d24e8ea-9446-49e9-8e86-deec6992c9d6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:09.627385 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1d24e8ea-9446-49e9-8e86-deec6992c9d6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:09.627823 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1d24e8ea-9446-49e9-8e86-deec6992c9d6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:09.628540 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1d24e8ea-9446-49e9-8e86-deec6992c9d6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:09.629008 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1d24e8ea-9446-49e9-8e86-deec6992c9d6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:09.629429 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1d24e8ea-9446-49e9-8e86-deec6992c9d6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:09.844357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-1d24e8ea-9446-49e9-8e86-deec6992c9d6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:09.852390 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-1d24e8ea-9446-49e9-8e86-deec6992c9d6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:09.853828 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-1d24e8ea-9446-49e9-8e86-deec6992c9d6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:09.855025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-1d24e8ea-9446-49e9-8e86-deec6992c9d6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1744 microversion: 2.20 time: 0.239876 Oct 24 08:59:09.855580 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 389/784] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:09 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1744 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:10.406044 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-830db697-6924-4f0f-8f8b-db6208a5ed6f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:10.406649 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-830db697-6924-4f0f-8f8b-db6208a5ed6f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:10.413060 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-830db697-6924-4f0f-8f8b-db6208a5ed6f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:10.413552 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-830db697-6924-4f0f-8f8b-db6208a5ed6f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:10.413992 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-830db697-6924-4f0f-8f8b-db6208a5ed6f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:10.414686 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-830db697-6924-4f0f-8f8b-db6208a5ed6f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:10.415137 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-830db697-6924-4f0f-8f8b-db6208a5ed6f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:10.415562 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-830db697-6924-4f0f-8f8b-db6208a5ed6f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:10.548088 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-830db697-6924-4f0f-8f8b-db6208a5ed6f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:10.556178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-830db697-6924-4f0f-8f8b-db6208a5ed6f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1364 microversion: 2.1 time: 0.154581 Oct 24 08:59:10.556750 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 396/785] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:10 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1364 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:10.882711 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-fff8dbde-fc58-4185-bb39-3fb8877824a8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:10.883321 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-fff8dbde-fc58-4185-bb39-3fb8877824a8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:10.891131 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fff8dbde-fc58-4185-bb39-3fb8877824a8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:10.891627 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fff8dbde-fc58-4185-bb39-3fb8877824a8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:10.892065 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fff8dbde-fc58-4185-bb39-3fb8877824a8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:10.892780 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fff8dbde-fc58-4185-bb39-3fb8877824a8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:10.893224 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fff8dbde-fc58-4185-bb39-3fb8877824a8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:10.893645 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fff8dbde-fc58-4185-bb39-3fb8877824a8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:11.036204 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fff8dbde-fc58-4185-bb39-3fb8877824a8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:11.044246 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fff8dbde-fc58-4185-bb39-3fb8877824a8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:11.045684 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fff8dbde-fc58-4185-bb39-3fb8877824a8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:11.047094 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-fff8dbde-fc58-4185-bb39-3fb8877824a8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1744 microversion: 2.20 time: 0.168126 Oct 24 08:59:11.047654 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 390/786] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:10 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1744 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:11.579728 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-19290d10-fff7-4390-9262-96eb87de7c00 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:11.580372 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-19290d10-fff7-4390-9262-96eb87de7c00 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:11.582453 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=2 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:11.583139 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=12 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:11.589948 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19290d10-fff7-4390-9262-96eb87de7c00 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:11.590535 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19290d10-fff7-4390-9262-96eb87de7c00 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:11.590975 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19290d10-fff7-4390-9262-96eb87de7c00 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:11.591667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19290d10-fff7-4390-9262-96eb87de7c00 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:11.592118 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19290d10-fff7-4390-9262-96eb87de7c00 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:11.592578 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19290d10-fff7-4390-9262-96eb87de7c00 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:11.722246 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-19290d10-fff7-4390-9262-96eb87de7c00 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:11.729903 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-19290d10-fff7-4390-9262-96eb87de7c00 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1364 microversion: 2.1 time: 0.155004 Oct 24 08:59:11.730484 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 397/787] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:11 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1364 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:12.076470 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-96bee64c-89f1-412d-bf46-a357ac5f4e05 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:12.077003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-96bee64c-89f1-412d-bf46-a357ac5f4e05 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:12.086180 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-96bee64c-89f1-412d-bf46-a357ac5f4e05 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:12.087034 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-96bee64c-89f1-412d-bf46-a357ac5f4e05 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:12.087472 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-96bee64c-89f1-412d-bf46-a357ac5f4e05 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:12.088334 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-96bee64c-89f1-412d-bf46-a357ac5f4e05 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:12.089175 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-96bee64c-89f1-412d-bf46-a357ac5f4e05 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:12.089738 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-96bee64c-89f1-412d-bf46-a357ac5f4e05 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:12.505285 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-96bee64c-89f1-412d-bf46-a357ac5f4e05 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:12.513659 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-96bee64c-89f1-412d-bf46-a357ac5f4e05 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:12.515705 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-96bee64c-89f1-412d-bf46-a357ac5f4e05 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:12.516959 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-96bee64c-89f1-412d-bf46-a357ac5f4e05 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1744 microversion: 2.20 time: 0.447124 Oct 24 08:59:12.517724 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 391/788] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:12 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1744 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:12.753765 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-0442cc26-dd6f-48e2-8f21-942ffd61f4d8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:12.754368 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-0442cc26-dd6f-48e2-8f21-942ffd61f4d8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:12.761057 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0442cc26-dd6f-48e2-8f21-942ffd61f4d8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:12.761554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0442cc26-dd6f-48e2-8f21-942ffd61f4d8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:12.761999 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0442cc26-dd6f-48e2-8f21-942ffd61f4d8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:12.762704 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0442cc26-dd6f-48e2-8f21-942ffd61f4d8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:12.763160 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0442cc26-dd6f-48e2-8f21-942ffd61f4d8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:12.763596 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0442cc26-dd6f-48e2-8f21-942ffd61f4d8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:12.898926 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0442cc26-dd6f-48e2-8f21-942ffd61f4d8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:12.907012 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-0442cc26-dd6f-48e2-8f21-942ffd61f4d8 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1364 microversion: 2.1 time: 0.157923 Oct 24 08:59:12.907583 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 398/789] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:12 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1364 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:13.541788 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ba11da38-86e3-4c86-8965-a308940c1e44 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:13.542392 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ba11da38-86e3-4c86-8965-a308940c1e44 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:13.549813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ba11da38-86e3-4c86-8965-a308940c1e44 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:13.550286 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ba11da38-86e3-4c86-8965-a308940c1e44 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:13.550716 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ba11da38-86e3-4c86-8965-a308940c1e44 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:13.551400 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ba11da38-86e3-4c86-8965-a308940c1e44 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:13.551844 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ba11da38-86e3-4c86-8965-a308940c1e44 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:13.552295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ba11da38-86e3-4c86-8965-a308940c1e44 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:13.712349 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ba11da38-86e3-4c86-8965-a308940c1e44 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:13.723041 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ba11da38-86e3-4c86-8965-a308940c1e44 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:13.724822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ba11da38-86e3-4c86-8965-a308940c1e44 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:13.726195 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ba11da38-86e3-4c86-8965-a308940c1e44 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1739 microversion: 2.20 time: 0.188451 Oct 24 08:59:13.727183 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 392/790] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:13 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1739 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:13.879817 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-f747df49-bbb5-45d0-956d-75ba9af0692e req-7cdc2ba1-010d-4992-974a-8a454737b036 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "240dec37-c7ff-4c59-8b5f-611a52575853", "name": "network-vif-plugged", "status": "completed", "tag": "86414159-5cb9-488f-8380-698e8b8e4c32"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:59:13.894775 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-f747df49-bbb5-45d0-956d-75ba9af0692e req-7cdc2ba1-010d-4992-974a-8a454737b036 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:13.895802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-f747df49-bbb5-45d0-956d-75ba9af0692e req-7cdc2ba1-010d-4992-974a-8a454737b036 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:13.896494 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-f747df49-bbb5-45d0-956d-75ba9af0692e req-7cdc2ba1-010d-4992-974a-8a454737b036 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:13.910765 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-f747df49-bbb5-45d0-956d-75ba9af0692e req-7cdc2ba1-010d-4992-974a-8a454737b036 service nova] Creating event network-vif-plugged:86414159-5cb9-488f-8380-698e8b8e4c32 for instance 240dec37-c7ff-4c59-8b5f-611a52575853 on hv-cinder-maste Oct 24 08:59:13.916173 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-f747df49-bbb5-45d0-956d-75ba9af0692e req-7cdc2ba1-010d-4992-974a-8a454737b036 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041783 Oct 24 08:59:13.919934 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 399/791] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:59:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:59:13.934716 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-80341a4a-fdd9-459a-b7ac-a2baabd309ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:13.935331 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-80341a4a-fdd9-459a-b7ac-a2baabd309ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:13.945662 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-80341a4a-fdd9-459a-b7ac-a2baabd309ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:13.946163 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-80341a4a-fdd9-459a-b7ac-a2baabd309ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:13.946606 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-80341a4a-fdd9-459a-b7ac-a2baabd309ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:13.947325 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-80341a4a-fdd9-459a-b7ac-a2baabd309ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:13.947778 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-80341a4a-fdd9-459a-b7ac-a2baabd309ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:13.948238 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-80341a4a-fdd9-459a-b7ac-a2baabd309ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:14.163522 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-80341a4a-fdd9-459a-b7ac-a2baabd309ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:14.173961 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-80341a4a-fdd9-459a-b7ac-a2baabd309ed tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1364 microversion: 2.1 time: 0.242023 Oct 24 08:59:14.180386 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 393/792] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:13 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1364 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:14.756576 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e03fc9fe-2284-4718-9915-5383bb306dca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:14.757764 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e03fc9fe-2284-4718-9915-5383bb306dca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:14.767059 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e03fc9fe-2284-4718-9915-5383bb306dca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:14.767769 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e03fc9fe-2284-4718-9915-5383bb306dca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:14.768465 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e03fc9fe-2284-4718-9915-5383bb306dca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:14.769319 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e03fc9fe-2284-4718-9915-5383bb306dca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:14.769859 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e03fc9fe-2284-4718-9915-5383bb306dca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:14.770362 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e03fc9fe-2284-4718-9915-5383bb306dca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:14.933888 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e03fc9fe-2284-4718-9915-5383bb306dca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:14.943288 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e03fc9fe-2284-4718-9915-5383bb306dca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:14.945179 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e03fc9fe-2284-4718-9915-5383bb306dca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:14.946489 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e03fc9fe-2284-4718-9915-5383bb306dca tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1739 microversion: 2.20 time: 0.195159 Oct 24 08:59:14.947128 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 400/793] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:14 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1739 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:15.198131 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-d42f697e-231e-47f0-a9ba-cada7a05b2e6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:15.198744 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-d42f697e-231e-47f0-a9ba-cada7a05b2e6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:15.205529 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d42f697e-231e-47f0-a9ba-cada7a05b2e6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:15.206022 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d42f697e-231e-47f0-a9ba-cada7a05b2e6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:15.206458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d42f697e-231e-47f0-a9ba-cada7a05b2e6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:15.207849 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d42f697e-231e-47f0-a9ba-cada7a05b2e6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:15.208336 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d42f697e-231e-47f0-a9ba-cada7a05b2e6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:15.208795 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d42f697e-231e-47f0-a9ba-cada7a05b2e6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:15.341631 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d42f697e-231e-47f0-a9ba-cada7a05b2e6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:15.350292 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-d42f697e-231e-47f0-a9ba-cada7a05b2e6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1364 microversion: 2.1 time: 0.156072 Oct 24 08:59:15.350846 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 394/794] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:15 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1364 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:15.974372 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c27656a8-2ec6-4164-84cb-3e1ba638b57c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:15.975017 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c27656a8-2ec6-4164-84cb-3e1ba638b57c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:15.981774 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c27656a8-2ec6-4164-84cb-3e1ba638b57c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:15.982634 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c27656a8-2ec6-4164-84cb-3e1ba638b57c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:15.983084 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c27656a8-2ec6-4164-84cb-3e1ba638b57c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:15.984721 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c27656a8-2ec6-4164-84cb-3e1ba638b57c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:15.985352 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c27656a8-2ec6-4164-84cb-3e1ba638b57c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:15.985784 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c27656a8-2ec6-4164-84cb-3e1ba638b57c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:16.133415 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c27656a8-2ec6-4164-84cb-3e1ba638b57c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:16.148504 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c27656a8-2ec6-4164-84cb-3e1ba638b57c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:16.149981 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c27656a8-2ec6-4164-84cb-3e1ba638b57c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:16.151176 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c27656a8-2ec6-4164-84cb-3e1ba638b57c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1739 microversion: 2.20 time: 0.180716 Oct 24 08:59:16.151741 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 401/795] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:15 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1739 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:16.374527 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-630d45b5-20a8-41fa-96b0-a8ed8d8952eb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:16.375147 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-630d45b5-20a8-41fa-96b0-a8ed8d8952eb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:16.382054 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-630d45b5-20a8-41fa-96b0-a8ed8d8952eb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:16.382544 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-630d45b5-20a8-41fa-96b0-a8ed8d8952eb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:16.382983 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-630d45b5-20a8-41fa-96b0-a8ed8d8952eb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:16.383717 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-630d45b5-20a8-41fa-96b0-a8ed8d8952eb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:16.384353 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-630d45b5-20a8-41fa-96b0-a8ed8d8952eb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:16.384954 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-630d45b5-20a8-41fa-96b0-a8ed8d8952eb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:16.389141 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:16.524721 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-630d45b5-20a8-41fa-96b0-a8ed8d8952eb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:16.535006 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-630d45b5-20a8-41fa-96b0-a8ed8d8952eb tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1554 microversion: 2.1 time: 0.164938 Oct 24 08:59:16.552157 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 395/796] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:16 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1554 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:16.560763 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c20ce67a-8cad-4a87-ae68-ec20967ca22f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:16.561295 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c20ce67a-8cad-4a87-ae68-ec20967ca22f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:16.568268 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c20ce67a-8cad-4a87-ae68-ec20967ca22f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:16.568782 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c20ce67a-8cad-4a87-ae68-ec20967ca22f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:16.569218 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c20ce67a-8cad-4a87-ae68-ec20967ca22f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:16.569927 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c20ce67a-8cad-4a87-ae68-ec20967ca22f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:16.570511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c20ce67a-8cad-4a87-ae68-ec20967ca22f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:16.571088 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c20ce67a-8cad-4a87-ae68-ec20967ca22f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:16.654228 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c20ce67a-8cad-4a87-ae68-ec20967ca22f tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.098085 Oct 24 08:59:16.656796 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 402/797] 10.222.0.39 () {58 vars in 1275 bytes} [Mon Oct 24 08:59:16 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853/os-interface => generated 284 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:59:17.178991 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-bc0e89e8-a8b7-494e-ad34-952e7f7b468b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:17.179864 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-bc0e89e8-a8b7-494e-ad34-952e7f7b468b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:17.186909 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc0e89e8-a8b7-494e-ad34-952e7f7b468b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:17.187760 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc0e89e8-a8b7-494e-ad34-952e7f7b468b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:17.188420 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc0e89e8-a8b7-494e-ad34-952e7f7b468b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:17.191691 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc0e89e8-a8b7-494e-ad34-952e7f7b468b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:17.192392 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc0e89e8-a8b7-494e-ad34-952e7f7b468b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:17.192998 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bc0e89e8-a8b7-494e-ad34-952e7f7b468b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:17.383090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-bc0e89e8-a8b7-494e-ad34-952e7f7b468b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:17.395645 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-bc0e89e8-a8b7-494e-ad34-952e7f7b468b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:17.397706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-bc0e89e8-a8b7-494e-ad34-952e7f7b468b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:17.399418 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-bc0e89e8-a8b7-494e-ad34-952e7f7b468b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1739 microversion: 2.20 time: 0.224859 Oct 24 08:59:17.416110 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 396/798] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:17 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1739 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:17.995557 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-55735089-5a54-4bae-8956-e6144f76dec1 req-49bd4d0d-1541-48f7-ba17-99ed89be12e9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "240dec37-c7ff-4c59-8b5f-611a52575853", "tag": "86414159-5cb9-488f-8380-698e8b8e4c32"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:59:18.009088 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-55735089-5a54-4bae-8956-e6144f76dec1 req-49bd4d0d-1541-48f7-ba17-99ed89be12e9 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:18.013220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-55735089-5a54-4bae-8956-e6144f76dec1 req-49bd4d0d-1541-48f7-ba17-99ed89be12e9 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:18.013754 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-55735089-5a54-4bae-8956-e6144f76dec1 req-49bd4d0d-1541-48f7-ba17-99ed89be12e9 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:18.031384 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-55735089-5a54-4bae-8956-e6144f76dec1 req-49bd4d0d-1541-48f7-ba17-99ed89be12e9 service nova] Creating event network-changed:86414159-5cb9-488f-8380-698e8b8e4c32 for instance 240dec37-c7ff-4c59-8b5f-611a52575853 on hv-cinder-maste Oct 24 08:59:18.038464 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-55735089-5a54-4bae-8956-e6144f76dec1 req-49bd4d0d-1541-48f7-ba17-99ed89be12e9 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047113 Oct 24 08:59:18.039725 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 403/799] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:59:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:59:18.426568 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-3e046c54-b145-403e-a69a-996f9b359cf6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:18.427193 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-3e046c54-b145-403e-a69a-996f9b359cf6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:18.435907 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3e046c54-b145-403e-a69a-996f9b359cf6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:18.436973 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3e046c54-b145-403e-a69a-996f9b359cf6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:18.440519 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3e046c54-b145-403e-a69a-996f9b359cf6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:18.443432 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3e046c54-b145-403e-a69a-996f9b359cf6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:18.444098 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3e046c54-b145-403e-a69a-996f9b359cf6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:18.444719 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3e046c54-b145-403e-a69a-996f9b359cf6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:18.725931 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-3e046c54-b145-403e-a69a-996f9b359cf6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:18.739589 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-3e046c54-b145-403e-a69a-996f9b359cf6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:18.741193 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-3e046c54-b145-403e-a69a-996f9b359cf6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:18.753049 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-3e046c54-b145-403e-a69a-996f9b359cf6 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1739 microversion: 2.20 time: 0.330481 Oct 24 08:59:18.753939 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 397/800] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:18 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1739 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:19.782294 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d76055bb-14cf-42fe-8afc-ccf7fe357ad4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:19.783240 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d76055bb-14cf-42fe-8afc-ccf7fe357ad4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:19.790848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d76055bb-14cf-42fe-8afc-ccf7fe357ad4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:19.791334 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d76055bb-14cf-42fe-8afc-ccf7fe357ad4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:19.791773 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d76055bb-14cf-42fe-8afc-ccf7fe357ad4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:19.794314 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d76055bb-14cf-42fe-8afc-ccf7fe357ad4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:19.794738 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d76055bb-14cf-42fe-8afc-ccf7fe357ad4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:19.795379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d76055bb-14cf-42fe-8afc-ccf7fe357ad4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:19.946905 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d76055bb-14cf-42fe-8afc-ccf7fe357ad4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:19.956341 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d76055bb-14cf-42fe-8afc-ccf7fe357ad4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:19.958223 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d76055bb-14cf-42fe-8afc-ccf7fe357ad4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:19.959522 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d76055bb-14cf-42fe-8afc-ccf7fe357ad4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1739 microversion: 2.20 time: 0.182626 Oct 24 08:59:19.960586 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 404/801] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:19 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1739 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:20.989477 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-92ef64a3-7cf8-4813-a695-13c42b5334d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:20.990867 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-92ef64a3-7cf8-4813-a695-13c42b5334d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:20.998058 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-92ef64a3-7cf8-4813-a695-13c42b5334d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:20.998810 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-92ef64a3-7cf8-4813-a695-13c42b5334d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:20.999449 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-92ef64a3-7cf8-4813-a695-13c42b5334d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:21.000409 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-92ef64a3-7cf8-4813-a695-13c42b5334d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:21.001056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-92ef64a3-7cf8-4813-a695-13c42b5334d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:21.001669 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-92ef64a3-7cf8-4813-a695-13c42b5334d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:21.147246 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-92ef64a3-7cf8-4813-a695-13c42b5334d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:21.157552 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-92ef64a3-7cf8-4813-a695-13c42b5334d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:21.159010 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-92ef64a3-7cf8-4813-a695-13c42b5334d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:21.160202 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-92ef64a3-7cf8-4813-a695-13c42b5334d8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1739 microversion: 2.20 time: 0.177095 Oct 24 08:59:21.160766 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 398/802] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:20 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1739 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:22.185995 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-0a734f8c-6140-4841-a441-0c16bfa0f430 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:22.187229 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-0a734f8c-6140-4841-a441-0c16bfa0f430 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:22.193811 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0a734f8c-6140-4841-a441-0c16bfa0f430 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:22.194303 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0a734f8c-6140-4841-a441-0c16bfa0f430 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:22.194743 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0a734f8c-6140-4841-a441-0c16bfa0f430 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:22.195449 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0a734f8c-6140-4841-a441-0c16bfa0f430 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:22.195909 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0a734f8c-6140-4841-a441-0c16bfa0f430 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:22.196365 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0a734f8c-6140-4841-a441-0c16bfa0f430 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:22.340500 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0a734f8c-6140-4841-a441-0c16bfa0f430 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:22.357096 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0a734f8c-6140-4841-a441-0c16bfa0f430 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:22.363370 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0a734f8c-6140-4841-a441-0c16bfa0f430 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:22.367777 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-0a734f8c-6140-4841-a441-0c16bfa0f430 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1739 microversion: 2.20 time: 0.185386 Oct 24 08:59:22.368887 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 405/803] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:22 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1739 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:59:23.394935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ce1802b0-4a52-412a-9ab2-d6ebff342004 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:23.395706 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ce1802b0-4a52-412a-9ab2-d6ebff342004 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:23.402385 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ce1802b0-4a52-412a-9ab2-d6ebff342004 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:23.403414 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ce1802b0-4a52-412a-9ab2-d6ebff342004 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:23.403997 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ce1802b0-4a52-412a-9ab2-d6ebff342004 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:23.406897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ce1802b0-4a52-412a-9ab2-d6ebff342004 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:23.407716 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ce1802b0-4a52-412a-9ab2-d6ebff342004 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:23.408345 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ce1802b0-4a52-412a-9ab2-d6ebff342004 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:23.602620 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ce1802b0-4a52-412a-9ab2-d6ebff342004 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:23.611870 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ce1802b0-4a52-412a-9ab2-d6ebff342004 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:23.614950 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ce1802b0-4a52-412a-9ab2-d6ebff342004 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:23.616396 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ce1802b0-4a52-412a-9ab2-d6ebff342004 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1739 microversion: 2.20 time: 0.225582 Oct 24 08:59:23.617540 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 399/804] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:23 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1739 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:24.641900 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e18a43d7-7b1d-40d6-9592-59c03b35f25f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:24.642538 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e18a43d7-7b1d-40d6-9592-59c03b35f25f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:24.649073 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e18a43d7-7b1d-40d6-9592-59c03b35f25f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:24.649551 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e18a43d7-7b1d-40d6-9592-59c03b35f25f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:24.649991 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e18a43d7-7b1d-40d6-9592-59c03b35f25f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:24.650704 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e18a43d7-7b1d-40d6-9592-59c03b35f25f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:24.651154 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e18a43d7-7b1d-40d6-9592-59c03b35f25f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:24.651578 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e18a43d7-7b1d-40d6-9592-59c03b35f25f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:24.783403 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e18a43d7-7b1d-40d6-9592-59c03b35f25f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:24.791713 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e18a43d7-7b1d-40d6-9592-59c03b35f25f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:24.793195 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e18a43d7-7b1d-40d6-9592-59c03b35f25f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:24.794456 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e18a43d7-7b1d-40d6-9592-59c03b35f25f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1739 microversion: 2.20 time: 0.156372 Oct 24 08:59:24.794895 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 406/805] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:24 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1739 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:25.819846 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0fdc765a-6efb-49be-a76b-3c83db65a75a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:25.820472 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-0fdc765a-6efb-49be-a76b-3c83db65a75a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:25.823010 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=56,nova_api:UPDATE=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:25.829099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fdc765a-6efb-49be-a76b-3c83db65a75a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:25.829596 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fdc765a-6efb-49be-a76b-3c83db65a75a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:25.830069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fdc765a-6efb-49be-a76b-3c83db65a75a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:25.830777 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fdc765a-6efb-49be-a76b-3c83db65a75a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:25.831234 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fdc765a-6efb-49be-a76b-3c83db65a75a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:25.831659 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fdc765a-6efb-49be-a76b-3c83db65a75a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:25.969756 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0fdc765a-6efb-49be-a76b-3c83db65a75a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:25.978150 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0fdc765a-6efb-49be-a76b-3c83db65a75a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:25.979606 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0fdc765a-6efb-49be-a76b-3c83db65a75a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:25.980815 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-0fdc765a-6efb-49be-a76b-3c83db65a75a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1739 microversion: 2.20 time: 0.164809 Oct 24 08:59:25.981395 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 400/806] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:25 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1739 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:27.007054 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-241e509c-73c0-4541-ae90-593b32e0ea91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:27.007664 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-241e509c-73c0-4541-ae90-593b32e0ea91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:27.010127 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=79,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:27.015970 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-241e509c-73c0-4541-ae90-593b32e0ea91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:27.016488 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-241e509c-73c0-4541-ae90-593b32e0ea91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:27.016928 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-241e509c-73c0-4541-ae90-593b32e0ea91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:27.017631 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-241e509c-73c0-4541-ae90-593b32e0ea91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:27.018088 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-241e509c-73c0-4541-ae90-593b32e0ea91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:27.018634 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-241e509c-73c0-4541-ae90-593b32e0ea91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:27.039912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=100 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:27.154313 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-241e509c-73c0-4541-ae90-593b32e0ea91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:27.164191 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-241e509c-73c0-4541-ae90-593b32e0ea91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:27.165824 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-241e509c-73c0-4541-ae90-593b32e0ea91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:27.166992 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-241e509c-73c0-4541-ae90-593b32e0ea91 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1739 microversion: 2.20 time: 0.163940 Oct 24 08:59:27.167573 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 407/807] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:27 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1739 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:28.193500 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-941260e3-af1d-4236-aa60-5f9c5db1e36d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:28.194064 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-941260e3-af1d-4236-aa60-5f9c5db1e36d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:28.200402 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-941260e3-af1d-4236-aa60-5f9c5db1e36d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:28.200882 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-941260e3-af1d-4236-aa60-5f9c5db1e36d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:28.201324 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-941260e3-af1d-4236-aa60-5f9c5db1e36d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:28.202635 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-941260e3-af1d-4236-aa60-5f9c5db1e36d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:28.203238 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-941260e3-af1d-4236-aa60-5f9c5db1e36d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:28.203664 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-941260e3-af1d-4236-aa60-5f9c5db1e36d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:28.337757 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-941260e3-af1d-4236-aa60-5f9c5db1e36d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:28.346655 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-941260e3-af1d-4236-aa60-5f9c5db1e36d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:28.348097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-941260e3-af1d-4236-aa60-5f9c5db1e36d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:28.349288 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-941260e3-af1d-4236-aa60-5f9c5db1e36d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1739 microversion: 2.20 time: 0.159785 Oct 24 08:59:28.349838 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 401/808] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:28 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1739 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:29.283798 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-0a2bc402-e4d6-4bf7-95be-30cf269ed099 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:29.284377 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-0a2bc402-e4d6-4bf7-95be-30cf269ed099 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:29.285564 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-5791d528-67ca-40d9-83b9-2e63e33ec81e req-08b95887-c5d1-4314-817b-a4b44ece96a4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "354e90c0-1ef5-43e0-8983-2067982a3f76", "tag": "4a9dc16c-b094-43ae-a495-1098e5c9f054"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:59:29.291055 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0a2bc402-e4d6-4bf7-95be-30cf269ed099 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:29.291552 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0a2bc402-e4d6-4bf7-95be-30cf269ed099 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:29.291997 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0a2bc402-e4d6-4bf7-95be-30cf269ed099 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:29.293082 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0a2bc402-e4d6-4bf7-95be-30cf269ed099 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:29.293944 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0a2bc402-e4d6-4bf7-95be-30cf269ed099 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:29.294400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0a2bc402-e4d6-4bf7-95be-30cf269ed099 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:29.298635 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-5791d528-67ca-40d9-83b9-2e63e33ec81e req-08b95887-c5d1-4314-817b-a4b44ece96a4 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:29.299438 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-5791d528-67ca-40d9-83b9-2e63e33ec81e req-08b95887-c5d1-4314-817b-a4b44ece96a4 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:29.300302 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-5791d528-67ca-40d9-83b9-2e63e33ec81e req-08b95887-c5d1-4314-817b-a4b44ece96a4 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:29.325782 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-0a2bc402-e4d6-4bf7-95be-30cf269ed099 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Going to try to terminate instance {{(pid=130395) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 08:59:29.328076 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-5791d528-67ca-40d9-83b9-2e63e33ec81e req-08b95887-c5d1-4314-817b-a4b44ece96a4 service nova] Creating event network-changed:4a9dc16c-b094-43ae-a495-1098e5c9f054 for instance 354e90c0-1ef5-43e0-8983-2067982a3f76 on hv-cinder-maste Oct 24 08:59:29.342836 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-5791d528-67ca-40d9-83b9-2e63e33ec81e req-08b95887-c5d1-4314-817b-a4b44ece96a4 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059858 Oct 24 08:59:29.347952 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 402/809] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:59:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:59:29.376116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5054db79-25ea-4543-bee1-6ae86dc81251 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:29.377231 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-5054db79-25ea-4543-bee1-6ae86dc81251 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:29.385870 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5054db79-25ea-4543-bee1-6ae86dc81251 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:29.386626 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5054db79-25ea-4543-bee1-6ae86dc81251 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:29.387233 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5054db79-25ea-4543-bee1-6ae86dc81251 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:29.388864 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5054db79-25ea-4543-bee1-6ae86dc81251 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:29.390080 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5054db79-25ea-4543-bee1-6ae86dc81251 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:29.391047 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5054db79-25ea-4543-bee1-6ae86dc81251 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:29.429034 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-0a2bc402-e4d6-4bf7-95be-30cf269ed099 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "DELETE /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 204 len: 0 microversion: 2.1 time: 0.149395 Oct 24 08:59:29.430687 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 408/810] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 08:59:29 2022] DELETE /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 08:59:29.462674 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7100d32c-f92f-417d-b4b3-576d6deacabf tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:29.463454 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7100d32c-f92f-417d-b4b3-576d6deacabf tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:29.470464 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7100d32c-f92f-417d-b4b3-576d6deacabf tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:29.470957 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7100d32c-f92f-417d-b4b3-576d6deacabf tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:29.471592 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7100d32c-f92f-417d-b4b3-576d6deacabf tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:29.473771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7100d32c-f92f-417d-b4b3-576d6deacabf tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:29.474430 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7100d32c-f92f-417d-b4b3-576d6deacabf tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:29.474864 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7100d32c-f92f-417d-b4b3-576d6deacabf tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:29.627912 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5054db79-25ea-4543-bee1-6ae86dc81251 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:29.636535 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5054db79-25ea-4543-bee1-6ae86dc81251 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:29.638277 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5054db79-25ea-4543-bee1-6ae86dc81251 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:29.640273 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5054db79-25ea-4543-bee1-6ae86dc81251 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1721 microversion: 2.20 time: 0.267937 Oct 24 08:59:29.641133 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 403/811] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:29 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1721 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:29.659811 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7100d32c-f92f-417d-b4b3-576d6deacabf tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:29.667829 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7100d32c-f92f-417d-b4b3-576d6deacabf tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1569 microversion: 2.1 time: 0.232719 Oct 24 08:59:29.668463 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 409/812] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:29 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1569 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:30.666740 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e148fa0e-de60-4868-9b9e-2e669796535b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:30.668069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e148fa0e-de60-4868-9b9e-2e669796535b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:30.675375 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e148fa0e-de60-4868-9b9e-2e669796535b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:30.675862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e148fa0e-de60-4868-9b9e-2e669796535b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:30.676339 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e148fa0e-de60-4868-9b9e-2e669796535b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:30.677043 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e148fa0e-de60-4868-9b9e-2e669796535b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:30.677523 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e148fa0e-de60-4868-9b9e-2e669796535b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:30.677963 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e148fa0e-de60-4868-9b9e-2e669796535b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:30.692093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-14f433aa-adb0-4e28-b5c6-1d13a1c90745 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:30.692732 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-14f433aa-adb0-4e28-b5c6-1d13a1c90745 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:30.699119 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-14f433aa-adb0-4e28-b5c6-1d13a1c90745 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:30.699595 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-14f433aa-adb0-4e28-b5c6-1d13a1c90745 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:30.700028 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-14f433aa-adb0-4e28-b5c6-1d13a1c90745 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:30.700753 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-14f433aa-adb0-4e28-b5c6-1d13a1c90745 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:30.701200 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-14f433aa-adb0-4e28-b5c6-1d13a1c90745 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:30.701783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-14f433aa-adb0-4e28-b5c6-1d13a1c90745 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:30.875127 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e148fa0e-de60-4868-9b9e-2e669796535b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:30.883366 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e148fa0e-de60-4868-9b9e-2e669796535b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:30.884941 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e148fa0e-de60-4868-9b9e-2e669796535b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:30.886105 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e148fa0e-de60-4868-9b9e-2e669796535b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1721 microversion: 2.20 time: 0.223227 Oct 24 08:59:30.886680 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 404/813] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:30 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1721 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:30.888489 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-14f433aa-adb0-4e28-b5c6-1d13a1c90745 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:30.896707 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-14f433aa-adb0-4e28-b5c6-1d13a1c90745 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1569 microversion: 2.1 time: 0.208427 Oct 24 08:59:30.897266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 410/814] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:30 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1569 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:31.914776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-32de676a-71f7-4ea7-a1c8-0c5513f1fb2b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:31.915875 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-32de676a-71f7-4ea7-a1c8-0c5513f1fb2b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:31.921755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c6b3fed7-342e-4546-801b-a18bcaa34114 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:31.922251 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c6b3fed7-342e-4546-801b-a18bcaa34114 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:31.925437 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32de676a-71f7-4ea7-a1c8-0c5513f1fb2b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:31.925930 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32de676a-71f7-4ea7-a1c8-0c5513f1fb2b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:31.926369 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32de676a-71f7-4ea7-a1c8-0c5513f1fb2b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:31.927086 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32de676a-71f7-4ea7-a1c8-0c5513f1fb2b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:31.927546 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32de676a-71f7-4ea7-a1c8-0c5513f1fb2b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:31.927973 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-32de676a-71f7-4ea7-a1c8-0c5513f1fb2b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:31.928103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c6b3fed7-342e-4546-801b-a18bcaa34114 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:31.928576 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c6b3fed7-342e-4546-801b-a18bcaa34114 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:31.929012 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c6b3fed7-342e-4546-801b-a18bcaa34114 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:31.929701 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c6b3fed7-342e-4546-801b-a18bcaa34114 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:31.930145 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c6b3fed7-342e-4546-801b-a18bcaa34114 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:31.930567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c6b3fed7-342e-4546-801b-a18bcaa34114 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:32.132308 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-32de676a-71f7-4ea7-a1c8-0c5513f1fb2b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:32.136470 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c6b3fed7-342e-4546-801b-a18bcaa34114 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:32.141155 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-32de676a-71f7-4ea7-a1c8-0c5513f1fb2b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:32.142635 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-32de676a-71f7-4ea7-a1c8-0c5513f1fb2b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:32.143813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-32de676a-71f7-4ea7-a1c8-0c5513f1fb2b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1721 microversion: 2.20 time: 0.233327 Oct 24 08:59:32.144400 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 405/815] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:31 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1721 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:32.144982 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c6b3fed7-342e-4546-801b-a18bcaa34114 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1569 microversion: 2.1 time: 0.228117 Oct 24 08:59:32.145540 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 411/816] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:31 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1569 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:33.168862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-fa902130-cf83-44d1-aab3-b3bf2822414a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:33.169444 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-fa902130-cf83-44d1-aab3-b3bf2822414a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:33.178822 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-5a3dc47a-c677-4afa-9895-bdceba09b34f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:33.179357 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-5a3dc47a-c677-4afa-9895-bdceba09b34f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:33.185826 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5a3dc47a-c677-4afa-9895-bdceba09b34f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:33.186062 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fa902130-cf83-44d1-aab3-b3bf2822414a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:33.186560 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fa902130-cf83-44d1-aab3-b3bf2822414a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:33.187138 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5a3dc47a-c677-4afa-9895-bdceba09b34f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:33.187434 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fa902130-cf83-44d1-aab3-b3bf2822414a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:33.187577 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5a3dc47a-c677-4afa-9895-bdceba09b34f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:33.188181 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fa902130-cf83-44d1-aab3-b3bf2822414a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:33.188490 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5a3dc47a-c677-4afa-9895-bdceba09b34f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:33.188844 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fa902130-cf83-44d1-aab3-b3bf2822414a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:33.188960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5a3dc47a-c677-4afa-9895-bdceba09b34f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:33.189275 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fa902130-cf83-44d1-aab3-b3bf2822414a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:33.189390 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5a3dc47a-c677-4afa-9895-bdceba09b34f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:33.523430 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fa902130-cf83-44d1-aab3-b3bf2822414a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:33.528169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-5a3dc47a-c677-4afa-9895-bdceba09b34f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:33.532933 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fa902130-cf83-44d1-aab3-b3bf2822414a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:33.534885 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fa902130-cf83-44d1-aab3-b3bf2822414a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:33.536262 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-fa902130-cf83-44d1-aab3-b3bf2822414a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1721 microversion: 2.20 time: 0.371291 Oct 24 08:59:33.538001 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-5a3dc47a-c677-4afa-9895-bdceba09b34f tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1569 microversion: 2.1 time: 0.362843 Oct 24 08:59:33.544979 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 406/817] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:33 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1721 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:33.545654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 412/818] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:33 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1569 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:34.561907 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c4cf4209-a0f2-4a16-8bfe-e5ae22a3ea87 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:34.562449 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b6f5ac4b-f18c-41f9-98b9-9bc466a5ecd1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:34.562817 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c4cf4209-a0f2-4a16-8bfe-e5ae22a3ea87 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:34.562962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b6f5ac4b-f18c-41f9-98b9-9bc466a5ecd1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:34.568932 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6f5ac4b-f18c-41f9-98b9-9bc466a5ecd1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:34.569124 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c4cf4209-a0f2-4a16-8bfe-e5ae22a3ea87 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:34.569421 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6f5ac4b-f18c-41f9-98b9-9bc466a5ecd1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:34.569635 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c4cf4209-a0f2-4a16-8bfe-e5ae22a3ea87 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:34.569860 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6f5ac4b-f18c-41f9-98b9-9bc466a5ecd1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:34.570074 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c4cf4209-a0f2-4a16-8bfe-e5ae22a3ea87 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:34.570560 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6f5ac4b-f18c-41f9-98b9-9bc466a5ecd1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:34.570837 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c4cf4209-a0f2-4a16-8bfe-e5ae22a3ea87 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:34.571076 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6f5ac4b-f18c-41f9-98b9-9bc466a5ecd1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:34.571292 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c4cf4209-a0f2-4a16-8bfe-e5ae22a3ea87 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:34.571511 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6f5ac4b-f18c-41f9-98b9-9bc466a5ecd1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:34.571765 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c4cf4209-a0f2-4a16-8bfe-e5ae22a3ea87 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:34.774113 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-b6f5ac4b-f18c-41f9-98b9-9bc466a5ecd1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:34.777277 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c4cf4209-a0f2-4a16-8bfe-e5ae22a3ea87 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:34.782846 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b6f5ac4b-f18c-41f9-98b9-9bc466a5ecd1 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1451 microversion: 2.1 time: 0.224572 Oct 24 08:59:34.783555 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 413/819] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:34 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1451 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:59:34.786209 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c4cf4209-a0f2-4a16-8bfe-e5ae22a3ea87 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:34.787625 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c4cf4209-a0f2-4a16-8bfe-e5ae22a3ea87 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:34.788802 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c4cf4209-a0f2-4a16-8bfe-e5ae22a3ea87 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1721 microversion: 2.20 time: 0.231040 Oct 24 08:59:34.789344 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 407/820] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:34 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1721 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:35.110473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-9846f191-d9fc-48b5-a730-a0f1a8aebc72 req-e3bc16fc-a337-400a-90df-438f54a7257d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e72acc7a-decb-46c7-9c51-f731e2c413b3", "name": "network-vif-unplugged", "status": "completed", "tag": "d0e79b5c-b764-405d-8c7a-d5032e3c72a8"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:59:35.120691 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-9846f191-d9fc-48b5-a730-a0f1a8aebc72 req-e3bc16fc-a337-400a-90df-438f54a7257d service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:35.121249 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-9846f191-d9fc-48b5-a730-a0f1a8aebc72 req-e3bc16fc-a337-400a-90df-438f54a7257d service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:35.121608 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-9846f191-d9fc-48b5-a730-a0f1a8aebc72 req-e3bc16fc-a337-400a-90df-438f54a7257d service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:35.136330 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-9846f191-d9fc-48b5-a730-a0f1a8aebc72 req-e3bc16fc-a337-400a-90df-438f54a7257d service nova] Creating event network-vif-unplugged:d0e79b5c-b764-405d-8c7a-d5032e3c72a8 for instance e72acc7a-decb-46c7-9c51-f731e2c413b3 on hv-cinder-maste Oct 24 08:59:35.142466 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-9846f191-d9fc-48b5-a730-a0f1a8aebc72 req-e3bc16fc-a337-400a-90df-438f54a7257d service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035820 Oct 24 08:59:35.143074 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 414/821] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:59:35 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 08:59:35.807312 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5f435c21-c67b-4609-9ee4-0dbf5bb785e7 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:35.808171 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-5f435c21-c67b-4609-9ee4-0dbf5bb785e7 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:35.815005 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-25ea2d31-4169-4247-bdcb-4331055be80d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:35.815715 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-25ea2d31-4169-4247-bdcb-4331055be80d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:35.815851 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5f435c21-c67b-4609-9ee4-0dbf5bb785e7 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:35.816298 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5f435c21-c67b-4609-9ee4-0dbf5bb785e7 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:35.816748 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5f435c21-c67b-4609-9ee4-0dbf5bb785e7 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:35.817457 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5f435c21-c67b-4609-9ee4-0dbf5bb785e7 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:35.817911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5f435c21-c67b-4609-9ee4-0dbf5bb785e7 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:35.818343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5f435c21-c67b-4609-9ee4-0dbf5bb785e7 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:35.821985 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-25ea2d31-4169-4247-bdcb-4331055be80d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:35.822471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-25ea2d31-4169-4247-bdcb-4331055be80d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:35.822911 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-25ea2d31-4169-4247-bdcb-4331055be80d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:35.823609 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-25ea2d31-4169-4247-bdcb-4331055be80d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:35.824060 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-25ea2d31-4169-4247-bdcb-4331055be80d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:35.824510 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-25ea2d31-4169-4247-bdcb-4331055be80d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:36.017583 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5f435c21-c67b-4609-9ee4-0dbf5bb785e7 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:36.019624 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-25ea2d31-4169-4247-bdcb-4331055be80d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:36.025436 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5f435c21-c67b-4609-9ee4-0dbf5bb785e7 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1451 microversion: 2.1 time: 0.222011 Oct 24 08:59:36.026003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 408/822] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:35 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1451 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:36.027714 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-25ea2d31-4169-4247-bdcb-4331055be80d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:36.029186 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-25ea2d31-4169-4247-bdcb-4331055be80d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:36.031070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-25ea2d31-4169-4247-bdcb-4331055be80d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1721 microversion: 2.20 time: 0.221559 Oct 24 08:59:36.031614 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 415/823] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:35 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1721 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:59:36.857014 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-682f8e80-3918-4309-82e2-9a9645517628 req-f2112fc7-d009-48ad-a325-bf1c56d35cd2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "354e90c0-1ef5-43e0-8983-2067982a3f76", "name": "network-vif-deleted", "tag": "4a9dc16c-b094-43ae-a495-1098e5c9f054"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:59:36.868991 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-682f8e80-3918-4309-82e2-9a9645517628 req-f2112fc7-d009-48ad-a325-bf1c56d35cd2 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:36.869462 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-682f8e80-3918-4309-82e2-9a9645517628 req-f2112fc7-d009-48ad-a325-bf1c56d35cd2 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:36.869880 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-682f8e80-3918-4309-82e2-9a9645517628 req-f2112fc7-d009-48ad-a325-bf1c56d35cd2 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:36.881763 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-682f8e80-3918-4309-82e2-9a9645517628 req-f2112fc7-d009-48ad-a325-bf1c56d35cd2 service nova] Creating event network-vif-deleted:4a9dc16c-b094-43ae-a495-1098e5c9f054 for instance 354e90c0-1ef5-43e0-8983-2067982a3f76 on hv-cinder-maste Oct 24 08:59:36.886655 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-682f8e80-3918-4309-82e2-9a9645517628 req-f2112fc7-d009-48ad-a325-bf1c56d35cd2 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034822 Oct 24 08:59:36.887230 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 409/824] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:59:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:59:37.050526 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b32d5608-47fe-44ca-bfc2-55d59b2c8bde tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:37.051154 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b32d5608-47fe-44ca-bfc2-55d59b2c8bde tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:37.057182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-3cc48823-a15e-49ae-9b98-28a0bc03b858 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:37.057747 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b32d5608-47fe-44ca-bfc2-55d59b2c8bde tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:37.057949 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-3cc48823-a15e-49ae-9b98-28a0bc03b858 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:37.058237 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b32d5608-47fe-44ca-bfc2-55d59b2c8bde tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:37.058754 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b32d5608-47fe-44ca-bfc2-55d59b2c8bde tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:37.059461 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b32d5608-47fe-44ca-bfc2-55d59b2c8bde tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:37.059940 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b32d5608-47fe-44ca-bfc2-55d59b2c8bde tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:37.060400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b32d5608-47fe-44ca-bfc2-55d59b2c8bde tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:37.064569 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3cc48823-a15e-49ae-9b98-28a0bc03b858 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:37.065055 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3cc48823-a15e-49ae-9b98-28a0bc03b858 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:37.065499 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3cc48823-a15e-49ae-9b98-28a0bc03b858 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:37.066198 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3cc48823-a15e-49ae-9b98-28a0bc03b858 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:37.066669 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3cc48823-a15e-49ae-9b98-28a0bc03b858 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:37.067100 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3cc48823-a15e-49ae-9b98-28a0bc03b858 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:37.141244 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-b32d5608-47fe-44ca-bfc2-55d59b2c8bde tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:37.151689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b32d5608-47fe-44ca-bfc2-55d59b2c8bde tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1383 microversion: 2.1 time: 0.105059 Oct 24 08:59:37.152770 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 416/825] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:37 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1383 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:37.238891 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-3cc48823-a15e-49ae-9b98-28a0bc03b858 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:37.248084 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-3cc48823-a15e-49ae-9b98-28a0bc03b858 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:37.250584 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-3cc48823-a15e-49ae-9b98-28a0bc03b858 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:37.251778 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-3cc48823-a15e-49ae-9b98-28a0bc03b858 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1724 microversion: 2.20 time: 0.199105 Oct 24 08:59:37.252406 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 410/826] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:37 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1724 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:59:37.277855 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7cd362f4-fa0a-45b5-a838-1716376da88d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "dbc774bb-63ee-4bb4-9b80-4ae4aaa238de"}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:59:37.287616 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7cd362f4-fa0a-45b5-a838-1716376da88d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:37.294683 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7cd362f4-fa0a-45b5-a838-1716376da88d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:37.295826 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7cd362f4-fa0a-45b5-a838-1716376da88d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:37.296344 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7cd362f4-fa0a-45b5-a838-1716376da88d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:37.297095 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7cd362f4-fa0a-45b5-a838-1716376da88d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:37.297547 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7cd362f4-fa0a-45b5-a838-1716376da88d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:37.297972 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7cd362f4-fa0a-45b5-a838-1716376da88d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:37.623717 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7cd362f4-fa0a-45b5-a838-1716376da88d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "POST /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.349383 Oct 24 08:59:37.630367 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 417/827] 10.222.0.39 () {62 vars in 1364 bytes} [Mon Oct 24 08:59:37 2022] POST /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3/os-volume_attachments => generated 188 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 24 08:59:37.636639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-145c6042-16b7-4733-ad46-cb31693fd59e tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:37.637142 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-145c6042-16b7-4733-ad46-cb31693fd59e tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:37.644077 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-145c6042-16b7-4733-ad46-cb31693fd59e tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:37.644616 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-145c6042-16b7-4733-ad46-cb31693fd59e tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:37.645058 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-145c6042-16b7-4733-ad46-cb31693fd59e tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:37.645760 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-145c6042-16b7-4733-ad46-cb31693fd59e tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:37.646209 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-145c6042-16b7-4733-ad46-cb31693fd59e tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:37.646809 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-145c6042-16b7-4733-ad46-cb31693fd59e tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:37.736107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.objects.instance [None req-145c6042-16b7-4733-ad46-cb31693fd59e tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lazy-loading 'services' on Instance uuid e7ae65b9-4038-4fad-b243-bbf157408181 {{(pid=130396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:59:37.798679 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-fa78913e-88e2-483d-b036-610e17aa55e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:37.799160 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-fa78913e-88e2-483d-b036-610e17aa55e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:37.806713 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fa78913e-88e2-483d-b036-610e17aa55e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:37.807447 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fa78913e-88e2-483d-b036-610e17aa55e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:37.807636 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fa78913e-88e2-483d-b036-610e17aa55e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:37.808382 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fa78913e-88e2-483d-b036-610e17aa55e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:37.808841 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fa78913e-88e2-483d-b036-610e17aa55e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:37.809272 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fa78913e-88e2-483d-b036-610e17aa55e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:37.822509 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-145c6042-16b7-4733-ad46-cb31693fd59e tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "DELETE /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181/os-volume_attachments/e8c3dfe6-2fa6-475f-8efe-2cf4bc2b8fa4" status: 202 len: 0 microversion: 2.1 time: 0.190115 Oct 24 08:59:37.823097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 411/828] 10.222.0.39 () {58 vars in 1416 bytes} [Mon Oct 24 08:59:37 2022] DELETE /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181/os-volume_attachments/e8c3dfe6-2fa6-475f-8efe-2cf4bc2b8fa4 => generated 0 bytes in 191 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 08:59:38.071443 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=87,nova_cell1:UPDATE=7,nova_cell1:INSERT=6 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:38.089284 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-fa78913e-88e2-483d-b036-610e17aa55e8 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "DELETE /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3/os-volume_attachments/dbc774bb-63ee-4bb4-9b80-4ae4aaa238de" status: 202 len: 0 microversion: 2.20 time: 0.295452 Oct 24 08:59:38.089970 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 418/829] 10.222.0.39 () {60 vars in 1457 bytes} [Mon Oct 24 08:59:37 2022] DELETE /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3/os-volume_attachments/dbc774bb-63ee-4bb4-9b80-4ae4aaa238de => generated 0 bytes in 297 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 24 08:59:38.155049 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-bf163092-d0cd-443f-8bc0-c9f362b70584 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:59:38.155719 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-bf163092-d0cd-443f-8bc0-c9f362b70584 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:38.162403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bf163092-d0cd-443f-8bc0-c9f362b70584 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:38.162854 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bf163092-d0cd-443f-8bc0-c9f362b70584 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:38.163283 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bf163092-d0cd-443f-8bc0-c9f362b70584 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:38.163991 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bf163092-d0cd-443f-8bc0-c9f362b70584 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:38.164474 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bf163092-d0cd-443f-8bc0-c9f362b70584 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:38.164892 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bf163092-d0cd-443f-8bc0-c9f362b70584 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:38.175856 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-19bed8d5-5279-4f68-ba33-0379e0466a36 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:38.176561 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-19bed8d5-5279-4f68-ba33-0379e0466a36 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:38.182860 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:38.183394 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19bed8d5-5279-4f68-ba33-0379e0466a36 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:38.183890 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19bed8d5-5279-4f68-ba33-0379e0466a36 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:38.184356 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19bed8d5-5279-4f68-ba33-0379e0466a36 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:38.185058 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19bed8d5-5279-4f68-ba33-0379e0466a36 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:38.185528 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19bed8d5-5279-4f68-ba33-0379e0466a36 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:38.185966 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19bed8d5-5279-4f68-ba33-0379e0466a36 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:38.216167 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-bf163092-d0cd-443f-8bc0-c9f362b70584 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=130396) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 24 08:59:38.218691 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19bed8d5-5279-4f68-ba33-0379e0466a36 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:38.219189 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19bed8d5-5279-4f68-ba33-0379e0466a36 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:38.219622 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19bed8d5-5279-4f68-ba33-0379e0466a36 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:38.220068 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-19bed8d5-5279-4f68-ba33-0379e0466a36 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lazy-loading 'fault' on Instance uuid 354e90c0-1ef5-43e0-8983-2067982a3f76 {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:59:38.262729 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-19bed8d5-5279-4f68-ba33-0379e0466a36 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82055ff074d842e88e27a441c32e9d01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29e4576f2b744d40878d65470e24c24c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:38.273173 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-19bed8d5-5279-4f68-ba33-0379e0466a36 tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 200 len: 1224 microversion: 2.1 time: 0.101406 Oct 24 08:59:38.273751 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 419/830] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:38 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 1224 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:59:38.291714 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-bf163092-d0cd-443f-8bc0-c9f362b70584 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "POST /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3/action" status: 202 len: 0 microversion: 2.20 time: 0.141932 Oct 24 08:59:38.292413 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 412/831] 10.222.0.39 () {62 vars in 1319 bytes} [Mon Oct 24 08:59:38 2022] POST /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3/action => generated 0 bytes in 144 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 24 08:59:38.301121 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-029a13e3-9e9e-4513-984c-59184672456a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:38.301766 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-029a13e3-9e9e-4513-984c-59184672456a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:38.307734 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-029a13e3-9e9e-4513-984c-59184672456a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:38.308271 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-029a13e3-9e9e-4513-984c-59184672456a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:38.308717 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-029a13e3-9e9e-4513-984c-59184672456a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:38.309644 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-029a13e3-9e9e-4513-984c-59184672456a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:38.310105 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-029a13e3-9e9e-4513-984c-59184672456a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:38.310537 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-029a13e3-9e9e-4513-984c-59184672456a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:38.449397 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-029a13e3-9e9e-4513-984c-59184672456a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:38.458355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-029a13e3-9e9e-4513-984c-59184672456a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:38.459808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-029a13e3-9e9e-4513-984c-59184672456a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:38.460995 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-029a13e3-9e9e-4513-984c-59184672456a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1672 microversion: 2.20 time: 0.163866 Oct 24 08:59:38.461547 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 420/832] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:38 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1672 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:39.298866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-23c8ea5e-ae16-4563-9b4c-97650e6b862d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:39.299475 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-23c8ea5e-ae16-4563-9b4c-97650e6b862d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] [instance: 354e90c0-1ef5-43e0-8983-2067982a3f76] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:39.305568 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-23c8ea5e-ae16-4563-9b4c-97650e6b862d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:39.306072 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-23c8ea5e-ae16-4563-9b4c-97650e6b862d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:39.306487 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-23c8ea5e-ae16-4563-9b4c-97650e6b862d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:39.307180 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-23c8ea5e-ae16-4563-9b4c-97650e6b862d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:39.307624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-23c8ea5e-ae16-4563-9b4c-97650e6b862d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:39.308043 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-23c8ea5e-ae16-4563-9b4c-97650e6b862d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:39.321670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-23c8ea5e-ae16-4563-9b4c-97650e6b862d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] HTTP exception thrown: Instance 354e90c0-1ef5-43e0-8983-2067982a3f76 could not be found. Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-23c8ea5e-ae16-4563-9b4c-97650e6b862d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Request method failure captured: Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 HTTP/1.1 Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlKR3FBSNa_vgjHNwRvmkS8viGFntWfuDSrJHHOfifsrkp7d4iTHTXct29J4pewTTKfd5iwh2KWYQIAadYP7O0S-bezaXoJaw8ubUT-gXjTNfZYFqSaAK9PD6W55MpcR0-PODBFJ9TFop8k1XWUvvtARsESNH_3QLMmhNh0bGvUmk3U Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 29e4576f2b744d40878d65470e24c24c Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-TestShelveInstance-1300225756 Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 08:59:39.323343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 08:59:39.325036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/29e4576f2b744d40878d65470e24c24c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_29e4576f2b744d40878d65470e24c24c", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/29e4576f2b744d40878d65470e24c24c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/29e4576f2b744d40878d65470e24c24c"}]}] Oct 24 08:59:39.325036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-TestShelveInstance-1300225756 Oct 24 08:59:39.325036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 29e4576f2b744d40878d65470e24c24c Oct 24 08:59:39.325036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-TestShelveInstance-1300225756 Oct 24 08:59:39.325036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-TestShelveInstance-1300225756-project Oct 24 08:59:39.325036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 08:59:39.325036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 08:59:39.325036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 82055ff074d842e88e27a441c32e9d01 Oct 24 08:59:39.325036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-TestShelveInstance-1300225756-project Oct 24 08:59:39.325036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 08:59:39.325036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance 354e90c0-1ef5-43e0-8983-2067982a3f76 could not be found. Oct 24 08:59:39.325036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': '354e90c0-1ef5-43e0-8983-2067982a3f76'} Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 354e90c0-1ef5-43e0-8983-2067982a3f76 could not be found. Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 08:59:39.325953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 354e90c0-1ef5-43e0-8983-2067982a3f76 could not be found. Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 354e90c0-1ef5-43e0-8983-2067982a3f76 could not be found. Oct 24 08:59:39.328107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 08:59:39.329542 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-23c8ea5e-ae16-4563-9b4c-97650e6b862d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Returning 404 to user: Instance 354e90c0-1ef5-43e0-8983-2067982a3f76 could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 08:59:39.329542 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-23c8ea5e-ae16-4563-9b4c-97650e6b862d tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76" status: 404 len: 111 microversion: 2.1 time: 0.030843 Oct 24 08:59:39.329542 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 413/833] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:39 2022] GET /compute/v2.1/servers/354e90c0-1ef5-43e0-8983-2067982a3f76 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 08:59:39.487240 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-aa8801fb-0322-43af-9c05-8704dcf580bf tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:39.487902 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-aa8801fb-0322-43af-9c05-8704dcf580bf tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:39.502653 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6c3284c0-8be2-4b4f-9b2d-f1a49fd2f13e tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] Calling method '.version_select at 0x7fdfedc4ff70>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:39.511515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa8801fb-0322-43af-9c05-8704dcf580bf tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:39.513353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa8801fb-0322-43af-9c05-8704dcf580bf tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:39.515474 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa8801fb-0322-43af-9c05-8704dcf580bf tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:39.518162 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa8801fb-0322-43af-9c05-8704dcf580bf tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:39.518880 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa8801fb-0322-43af-9c05-8704dcf580bf tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:39.519391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa8801fb-0322-43af-9c05-8704dcf580bf tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:39.525500 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6c3284c0-8be2-4b4f-9b2d-f1a49fd2f13e tempest-TestShelveInstance-1300225756 tempest-TestShelveInstance-1300225756-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1609252738" status: 202 len: 0 microversion: 2.1 time: 0.027638 Oct 24 08:59:39.526663 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 414/834] 10.222.0.39 () {58 vars in 1254 bytes} [Mon Oct 24 08:59:39 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1609252738 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 08:59:39.753759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-aa8801fb-0322-43af-9c05-8704dcf580bf tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:39.762126 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-aa8801fb-0322-43af-9c05-8704dcf580bf tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:39.763838 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-aa8801fb-0322-43af-9c05-8704dcf580bf tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:39.765328 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-aa8801fb-0322-43af-9c05-8704dcf580bf tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.282364 Oct 24 08:59:39.765927 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 421/835] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:39 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:40.031257 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-249dc02b-85fd-4900-b764-71a29f49e5d6 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:40.031768 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-249dc02b-85fd-4900-b764-71a29f49e5d6 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:40.038821 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-249dc02b-85fd-4900-b764-71a29f49e5d6 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:40.039313 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-249dc02b-85fd-4900-b764-71a29f49e5d6 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:40.039745 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-249dc02b-85fd-4900-b764-71a29f49e5d6 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:40.040485 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-249dc02b-85fd-4900-b764-71a29f49e5d6 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:40.040939 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-249dc02b-85fd-4900-b764-71a29f49e5d6 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:40.041367 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-249dc02b-85fd-4900-b764-71a29f49e5d6 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:40.071517 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-249dc02b-85fd-4900-b764-71a29f49e5d6 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Going to try to terminate instance {{(pid=130396) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 08:59:40.309004 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-249dc02b-85fd-4900-b764-71a29f49e5d6 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "DELETE /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 204 len: 0 microversion: 2.1 time: 0.281482 Oct 24 08:59:40.309907 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 415/836] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 08:59:40 2022] DELETE /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 08:59:40.318774 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-44b23d0e-4d9b-453d-9ee0-e1ed14a79904 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:40.319383 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-44b23d0e-4d9b-453d-9ee0-e1ed14a79904 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:40.325328 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-44b23d0e-4d9b-453d-9ee0-e1ed14a79904 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:40.325813 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-44b23d0e-4d9b-453d-9ee0-e1ed14a79904 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:40.326514 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-44b23d0e-4d9b-453d-9ee0-e1ed14a79904 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:40.327100 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-44b23d0e-4d9b-453d-9ee0-e1ed14a79904 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:40.327757 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-44b23d0e-4d9b-453d-9ee0-e1ed14a79904 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:40.328211 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-44b23d0e-4d9b-453d-9ee0-e1ed14a79904 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:40.613322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-44b23d0e-4d9b-453d-9ee0-e1ed14a79904 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:40.621344 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-44b23d0e-4d9b-453d-9ee0-e1ed14a79904 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1515 microversion: 2.1 time: 0.306402 Oct 24 08:59:40.621921 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 422/837] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:40 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1515 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:59:40.793294 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5b530835-9d86-4a2e-9449-87d0f4ddd157 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:40.793993 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-5b530835-9d86-4a2e-9449-87d0f4ddd157 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:40.800603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5b530835-9d86-4a2e-9449-87d0f4ddd157 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:40.801090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5b530835-9d86-4a2e-9449-87d0f4ddd157 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:40.801528 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5b530835-9d86-4a2e-9449-87d0f4ddd157 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:40.802227 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5b530835-9d86-4a2e-9449-87d0f4ddd157 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:40.802681 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5b530835-9d86-4a2e-9449-87d0f4ddd157 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:40.803110 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5b530835-9d86-4a2e-9449-87d0f4ddd157 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:41.015259 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5b530835-9d86-4a2e-9449-87d0f4ddd157 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:41.023307 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5b530835-9d86-4a2e-9449-87d0f4ddd157 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:41.024766 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5b530835-9d86-4a2e-9449-87d0f4ddd157 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:41.025962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5b530835-9d86-4a2e-9449-87d0f4ddd157 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.236964 Oct 24 08:59:41.026501 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 416/838] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:40 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:41.648703 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a5389c46-6f2a-4006-a426-35c9e3b59edd tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:41.650083 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a5389c46-6f2a-4006-a426-35c9e3b59edd tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:41.659453 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5389c46-6f2a-4006-a426-35c9e3b59edd tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:41.660359 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5389c46-6f2a-4006-a426-35c9e3b59edd tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:41.661571 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5389c46-6f2a-4006-a426-35c9e3b59edd tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:41.663070 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5389c46-6f2a-4006-a426-35c9e3b59edd tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:41.663761 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5389c46-6f2a-4006-a426-35c9e3b59edd tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:41.664933 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5389c46-6f2a-4006-a426-35c9e3b59edd tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:41.953832 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a5389c46-6f2a-4006-a426-35c9e3b59edd tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:41.964391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a5389c46-6f2a-4006-a426-35c9e3b59edd tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1515 microversion: 2.1 time: 0.320170 Oct 24 08:59:41.965334 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 423/839] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:41 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1515 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:42.053503 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-cdae1549-ca1b-41d6-b246-acfe750fef1f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:42.054113 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-cdae1549-ca1b-41d6-b246-acfe750fef1f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:42.061243 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cdae1549-ca1b-41d6-b246-acfe750fef1f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:42.062435 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cdae1549-ca1b-41d6-b246-acfe750fef1f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:42.063013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cdae1549-ca1b-41d6-b246-acfe750fef1f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:42.063720 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cdae1549-ca1b-41d6-b246-acfe750fef1f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:42.064195 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cdae1549-ca1b-41d6-b246-acfe750fef1f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:42.064700 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cdae1549-ca1b-41d6-b246-acfe750fef1f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:42.298381 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-cdae1549-ca1b-41d6-b246-acfe750fef1f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:42.308549 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-cdae1549-ca1b-41d6-b246-acfe750fef1f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:42.310256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-cdae1549-ca1b-41d6-b246-acfe750fef1f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:42.311777 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-cdae1549-ca1b-41d6-b246-acfe750fef1f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.262312 Oct 24 08:59:42.312463 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 417/840] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:42 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:59:42.993341 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-9157dc81-9ce5-4a4c-a951-1b23f8ff04d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:42.993998 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-9157dc81-9ce5-4a4c-a951-1b23f8ff04d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:43.022978 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9157dc81-9ce5-4a4c-a951-1b23f8ff04d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:43.023479 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9157dc81-9ce5-4a4c-a951-1b23f8ff04d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:43.023927 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9157dc81-9ce5-4a4c-a951-1b23f8ff04d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:43.024697 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9157dc81-9ce5-4a4c-a951-1b23f8ff04d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:43.025163 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9157dc81-9ce5-4a4c-a951-1b23f8ff04d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:43.025597 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-9157dc81-9ce5-4a4c-a951-1b23f8ff04d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:43.267114 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-c56c16cd-274f-419d-8501-99008f56b8b8 req-0a779f7d-0b2e-4403-8f3e-4e3a81b380b9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e7ae65b9-4038-4fad-b243-bbf157408181", "name": "network-vif-deleted", "tag": "ca181843-9f26-4a98-bbf5-490b90538167"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 08:59:43.285404 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-c56c16cd-274f-419d-8501-99008f56b8b8 req-0a779f7d-0b2e-4403-8f3e-4e3a81b380b9 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:43.287517 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-c56c16cd-274f-419d-8501-99008f56b8b8 req-0a779f7d-0b2e-4403-8f3e-4e3a81b380b9 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:43.288582 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-c56c16cd-274f-419d-8501-99008f56b8b8 req-0a779f7d-0b2e-4403-8f3e-4e3a81b380b9 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:43.290003 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-9157dc81-9ce5-4a4c-a951-1b23f8ff04d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:43.300146 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-9157dc81-9ce5-4a4c-a951-1b23f8ff04d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1473 microversion: 2.1 time: 0.314179 Oct 24 08:59:43.301200 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 424/841] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:42 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1473 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:43.307818 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-c56c16cd-274f-419d-8501-99008f56b8b8 req-0a779f7d-0b2e-4403-8f3e-4e3a81b380b9 service nova] Creating event network-vif-deleted:ca181843-9f26-4a98-bbf5-490b90538167 for instance e7ae65b9-4038-4fad-b243-bbf157408181 on hv-cinder-maste Oct 24 08:59:43.313525 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-c56c16cd-274f-419d-8501-99008f56b8b8 req-0a779f7d-0b2e-4403-8f3e-4e3a81b380b9 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050426 Oct 24 08:59:43.315768 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 418/842] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 08:59:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 08:59:43.339695 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-198aace6-1f79-422d-83c5-947dfa1f9509 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:43.340528 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-198aace6-1f79-422d-83c5-947dfa1f9509 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:43.348460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-198aace6-1f79-422d-83c5-947dfa1f9509 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:43.349000 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-198aace6-1f79-422d-83c5-947dfa1f9509 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:43.349501 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-198aace6-1f79-422d-83c5-947dfa1f9509 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:43.350278 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-198aace6-1f79-422d-83c5-947dfa1f9509 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:43.350937 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-198aace6-1f79-422d-83c5-947dfa1f9509 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:43.357176 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-198aace6-1f79-422d-83c5-947dfa1f9509 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:43.617087 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-198aace6-1f79-422d-83c5-947dfa1f9509 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:43.633554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-198aace6-1f79-422d-83c5-947dfa1f9509 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:43.636980 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-198aace6-1f79-422d-83c5-947dfa1f9509 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:43.639417 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-198aace6-1f79-422d-83c5-947dfa1f9509 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.304411 Oct 24 08:59:43.640947 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 425/843] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:43 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:44.331471 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f0a342e1-a946-4ccd-8f33-3b55b5ce3c90 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:44.333385 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f0a342e1-a946-4ccd-8f33-3b55b5ce3c90 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:44.350435 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f0a342e1-a946-4ccd-8f33-3b55b5ce3c90 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:44.350928 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f0a342e1-a946-4ccd-8f33-3b55b5ce3c90 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:44.351533 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f0a342e1-a946-4ccd-8f33-3b55b5ce3c90 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:44.352285 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f0a342e1-a946-4ccd-8f33-3b55b5ce3c90 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:44.352899 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f0a342e1-a946-4ccd-8f33-3b55b5ce3c90 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:44.353320 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f0a342e1-a946-4ccd-8f33-3b55b5ce3c90 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:44.442229 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f0a342e1-a946-4ccd-8f33-3b55b5ce3c90 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:44.450225 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f0a342e1-a946-4ccd-8f33-3b55b5ce3c90 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1473 microversion: 2.1 time: 0.125790 Oct 24 08:59:44.450811 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 419/844] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:44 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1473 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:44.667019 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d27e115a-b71f-4e87-b9d8-f9e54516b38a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:44.668418 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d27e115a-b71f-4e87-b9d8-f9e54516b38a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:44.678894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d27e115a-b71f-4e87-b9d8-f9e54516b38a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:44.679388 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d27e115a-b71f-4e87-b9d8-f9e54516b38a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:44.679829 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d27e115a-b71f-4e87-b9d8-f9e54516b38a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:44.680779 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d27e115a-b71f-4e87-b9d8-f9e54516b38a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:44.681503 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d27e115a-b71f-4e87-b9d8-f9e54516b38a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:44.682163 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d27e115a-b71f-4e87-b9d8-f9e54516b38a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:44.849160 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d27e115a-b71f-4e87-b9d8-f9e54516b38a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:44.859123 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d27e115a-b71f-4e87-b9d8-f9e54516b38a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:44.860906 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d27e115a-b71f-4e87-b9d8-f9e54516b38a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:44.862519 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d27e115a-b71f-4e87-b9d8-f9e54516b38a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.199407 Oct 24 08:59:44.872510 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 426/845] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:44 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:45.477456 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-006197cc-005a-428a-8504-65464f5f696c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:45.478910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-006197cc-005a-428a-8504-65464f5f696c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:45.490177 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-006197cc-005a-428a-8504-65464f5f696c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:45.491338 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-006197cc-005a-428a-8504-65464f5f696c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:45.492183 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-006197cc-005a-428a-8504-65464f5f696c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:45.493824 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-006197cc-005a-428a-8504-65464f5f696c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:45.494648 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-006197cc-005a-428a-8504-65464f5f696c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:45.495224 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-006197cc-005a-428a-8504-65464f5f696c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:45.580912 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-006197cc-005a-428a-8504-65464f5f696c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:45.589187 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-006197cc-005a-428a-8504-65464f5f696c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1473 microversion: 2.1 time: 0.118193 Oct 24 08:59:45.589757 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 420/846] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:45 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1473 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:45.890323 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-384455ed-9a45-4aa6-8bfe-6385196c740a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:45.890861 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-384455ed-9a45-4aa6-8bfe-6385196c740a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:45.898219 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-384455ed-9a45-4aa6-8bfe-6385196c740a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:45.898712 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-384455ed-9a45-4aa6-8bfe-6385196c740a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:45.899153 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-384455ed-9a45-4aa6-8bfe-6385196c740a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:45.899869 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-384455ed-9a45-4aa6-8bfe-6385196c740a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:45.900354 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-384455ed-9a45-4aa6-8bfe-6385196c740a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:45.900787 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-384455ed-9a45-4aa6-8bfe-6385196c740a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:46.085621 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-384455ed-9a45-4aa6-8bfe-6385196c740a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:46.101470 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-384455ed-9a45-4aa6-8bfe-6385196c740a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:46.103356 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-384455ed-9a45-4aa6-8bfe-6385196c740a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:46.109865 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-384455ed-9a45-4aa6-8bfe-6385196c740a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.224263 Oct 24 08:59:46.128361 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 427/847] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:45 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:46.615606 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0a30fe44-c05b-403f-977d-d325eb224552 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:46.616331 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-0a30fe44-c05b-403f-977d-d325eb224552 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:46.637040 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a30fe44-c05b-403f-977d-d325eb224552 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:46.637552 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a30fe44-c05b-403f-977d-d325eb224552 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:46.638002 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a30fe44-c05b-403f-977d-d325eb224552 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:46.638798 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a30fe44-c05b-403f-977d-d325eb224552 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:46.639257 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a30fe44-c05b-403f-977d-d325eb224552 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:46.639684 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0a30fe44-c05b-403f-977d-d325eb224552 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:46.786492 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0a30fe44-c05b-403f-977d-d325eb224552 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:46.796538 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-0a30fe44-c05b-403f-977d-d325eb224552 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1473 microversion: 2.1 time: 0.185804 Oct 24 08:59:46.802199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 421/848] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:46 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1473 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:47.137920 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-0fbfa5a2-8674-408f-9e79-e038ee964684 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:47.139007 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-0fbfa5a2-8674-408f-9e79-e038ee964684 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:47.145975 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0fbfa5a2-8674-408f-9e79-e038ee964684 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:47.146741 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0fbfa5a2-8674-408f-9e79-e038ee964684 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:47.147178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0fbfa5a2-8674-408f-9e79-e038ee964684 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:47.147885 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0fbfa5a2-8674-408f-9e79-e038ee964684 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:47.150071 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0fbfa5a2-8674-408f-9e79-e038ee964684 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:47.150567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0fbfa5a2-8674-408f-9e79-e038ee964684 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:47.390079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0fbfa5a2-8674-408f-9e79-e038ee964684 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:47.401913 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0fbfa5a2-8674-408f-9e79-e038ee964684 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:47.406412 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0fbfa5a2-8674-408f-9e79-e038ee964684 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:47.409341 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-0fbfa5a2-8674-408f-9e79-e038ee964684 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.274361 Oct 24 08:59:47.411918 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 428/849] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:47 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:47.822501 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2eb46def-1306-48bb-9534-a0ff23461b56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:47.823030 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2eb46def-1306-48bb-9534-a0ff23461b56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:47.832003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2eb46def-1306-48bb-9534-a0ff23461b56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:47.844099 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2eb46def-1306-48bb-9534-a0ff23461b56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:47.844942 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2eb46def-1306-48bb-9534-a0ff23461b56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:47.845938 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2eb46def-1306-48bb-9534-a0ff23461b56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:47.846610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2eb46def-1306-48bb-9534-a0ff23461b56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:47.847268 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2eb46def-1306-48bb-9534-a0ff23461b56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:48.069407 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2eb46def-1306-48bb-9534-a0ff23461b56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:48.080483 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2eb46def-1306-48bb-9534-a0ff23461b56 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1473 microversion: 2.1 time: 0.262184 Oct 24 08:59:48.081805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 422/850] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:47 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1473 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:48.444919 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-15e63c74-3c81-47af-87c4-453ca9fe67f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:48.445527 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-15e63c74-3c81-47af-87c4-453ca9fe67f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:48.464740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-15e63c74-3c81-47af-87c4-453ca9fe67f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:48.465323 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-15e63c74-3c81-47af-87c4-453ca9fe67f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:48.465983 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-15e63c74-3c81-47af-87c4-453ca9fe67f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:48.467140 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-15e63c74-3c81-47af-87c4-453ca9fe67f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:48.468116 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-15e63c74-3c81-47af-87c4-453ca9fe67f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:48.468621 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-15e63c74-3c81-47af-87c4-453ca9fe67f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:48.665188 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-15e63c74-3c81-47af-87c4-453ca9fe67f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:48.676377 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-15e63c74-3c81-47af-87c4-453ca9fe67f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:48.678125 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-15e63c74-3c81-47af-87c4-453ca9fe67f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:48.679512 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-15e63c74-3c81-47af-87c4-453ca9fe67f3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.238430 Oct 24 08:59:48.687280 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 429/851] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:48 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:49.107285 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-30b54a05-2878-4630-9944-69941788ac13 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:49.109158 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-30b54a05-2878-4630-9944-69941788ac13 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:49.129114 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-30b54a05-2878-4630-9944-69941788ac13 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:49.133603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-30b54a05-2878-4630-9944-69941788ac13 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:49.134902 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-30b54a05-2878-4630-9944-69941788ac13 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:49.137357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-30b54a05-2878-4630-9944-69941788ac13 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:49.141085 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-30b54a05-2878-4630-9944-69941788ac13 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:49.141801 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-30b54a05-2878-4630-9944-69941788ac13 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:49.247059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-30b54a05-2878-4630-9944-69941788ac13 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:49.256006 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-30b54a05-2878-4630-9944-69941788ac13 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1473 microversion: 2.1 time: 0.153812 Oct 24 08:59:49.256617 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 423/852] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:49 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1473 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:49.705769 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-544c6fb3-6da0-4517-b22e-1bf308f4936f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:49.707163 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-544c6fb3-6da0-4517-b22e-1bf308f4936f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:49.714207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-544c6fb3-6da0-4517-b22e-1bf308f4936f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:49.714721 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-544c6fb3-6da0-4517-b22e-1bf308f4936f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:49.715223 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-544c6fb3-6da0-4517-b22e-1bf308f4936f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:49.715883 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-544c6fb3-6da0-4517-b22e-1bf308f4936f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:49.716363 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-544c6fb3-6da0-4517-b22e-1bf308f4936f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:49.717209 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-544c6fb3-6da0-4517-b22e-1bf308f4936f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:49.864753 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-544c6fb3-6da0-4517-b22e-1bf308f4936f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:49.873847 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-544c6fb3-6da0-4517-b22e-1bf308f4936f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:49.875530 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-544c6fb3-6da0-4517-b22e-1bf308f4936f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:49.877144 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-544c6fb3-6da0-4517-b22e-1bf308f4936f tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.175644 Oct 24 08:59:49.889258 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 430/853] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:49 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:50.291468 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-05045f08-ecf4-4ff6-baea-a1dd1797b03a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:50.292075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-05045f08-ecf4-4ff6-baea-a1dd1797b03a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:50.299933 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05045f08-ecf4-4ff6-baea-a1dd1797b03a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:50.300491 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05045f08-ecf4-4ff6-baea-a1dd1797b03a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:50.301107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05045f08-ecf4-4ff6-baea-a1dd1797b03a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:50.301810 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05045f08-ecf4-4ff6-baea-a1dd1797b03a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:50.304679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05045f08-ecf4-4ff6-baea-a1dd1797b03a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:50.305767 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-05045f08-ecf4-4ff6-baea-a1dd1797b03a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:50.389437 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-05045f08-ecf4-4ff6-baea-a1dd1797b03a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:50.397554 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-05045f08-ecf4-4ff6-baea-a1dd1797b03a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1473 microversion: 2.1 time: 0.110070 Oct 24 08:59:50.398105 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 424/854] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:50 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1473 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:50.907607 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-24c2bb10-132d-4faa-8082-53ee3b4d0943 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:50.909353 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-24c2bb10-132d-4faa-8082-53ee3b4d0943 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:50.919372 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-24c2bb10-132d-4faa-8082-53ee3b4d0943 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:50.921375 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-24c2bb10-132d-4faa-8082-53ee3b4d0943 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:50.922484 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-24c2bb10-132d-4faa-8082-53ee3b4d0943 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:50.927249 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-24c2bb10-132d-4faa-8082-53ee3b4d0943 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:50.927249 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-24c2bb10-132d-4faa-8082-53ee3b4d0943 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:50.927249 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-24c2bb10-132d-4faa-8082-53ee3b4d0943 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:51.146031 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-24c2bb10-132d-4faa-8082-53ee3b4d0943 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:51.157202 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-24c2bb10-132d-4faa-8082-53ee3b4d0943 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:51.158674 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-24c2bb10-132d-4faa-8082-53ee3b4d0943 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:51.159876 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-24c2bb10-132d-4faa-8082-53ee3b4d0943 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.257704 Oct 24 08:59:51.160541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 431/855] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:50 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:51.421883 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-557c3de7-a6d7-41e1-b725-9b3792229137 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:51.422561 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-557c3de7-a6d7-41e1-b725-9b3792229137 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:51.429580 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-557c3de7-a6d7-41e1-b725-9b3792229137 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:51.430378 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-557c3de7-a6d7-41e1-b725-9b3792229137 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:51.430938 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-557c3de7-a6d7-41e1-b725-9b3792229137 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:51.431722 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-557c3de7-a6d7-41e1-b725-9b3792229137 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:51.432299 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-557c3de7-a6d7-41e1-b725-9b3792229137 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:51.432793 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-557c3de7-a6d7-41e1-b725-9b3792229137 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:51.551546 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-557c3de7-a6d7-41e1-b725-9b3792229137 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:51.567422 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-557c3de7-a6d7-41e1-b725-9b3792229137 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1473 microversion: 2.1 time: 0.149783 Oct 24 08:59:51.581779 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 425/856] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:51 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1473 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:52.187875 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7edd3b3a-1147-4a4a-bb8b-f238945edb4d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:52.189243 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7edd3b3a-1147-4a4a-bb8b-f238945edb4d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:52.202457 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7edd3b3a-1147-4a4a-bb8b-f238945edb4d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:52.203302 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7edd3b3a-1147-4a4a-bb8b-f238945edb4d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:52.203754 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7edd3b3a-1147-4a4a-bb8b-f238945edb4d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:52.210927 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7edd3b3a-1147-4a4a-bb8b-f238945edb4d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:52.211645 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7edd3b3a-1147-4a4a-bb8b-f238945edb4d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:52.212333 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7edd3b3a-1147-4a4a-bb8b-f238945edb4d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:52.489791 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7edd3b3a-1147-4a4a-bb8b-f238945edb4d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:52.503824 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7edd3b3a-1147-4a4a-bb8b-f238945edb4d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:52.507154 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7edd3b3a-1147-4a4a-bb8b-f238945edb4d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:52.515724 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7edd3b3a-1147-4a4a-bb8b-f238945edb4d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.334130 Oct 24 08:59:52.516442 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 432/857] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:52 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:52.592230 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-d0e07c7a-d4b2-4228-8f93-835a3f39d592 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:52.593082 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-d0e07c7a-d4b2-4228-8f93-835a3f39d592 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:52.611781 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d0e07c7a-d4b2-4228-8f93-835a3f39d592 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:52.612600 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d0e07c7a-d4b2-4228-8f93-835a3f39d592 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:52.613219 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d0e07c7a-d4b2-4228-8f93-835a3f39d592 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:52.614453 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d0e07c7a-d4b2-4228-8f93-835a3f39d592 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:52.615069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d0e07c7a-d4b2-4228-8f93-835a3f39d592 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:52.615654 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d0e07c7a-d4b2-4228-8f93-835a3f39d592 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:52.745963 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d0e07c7a-d4b2-4228-8f93-835a3f39d592 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:52.756468 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-d0e07c7a-d4b2-4228-8f93-835a3f39d592 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1473 microversion: 2.1 time: 0.168768 Oct 24 08:59:52.757372 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 426/858] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:52 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1473 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:53.545375 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c4aa7b87-711d-4214-9c3f-120d8d48d664 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:53.546025 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c4aa7b87-711d-4214-9c3f-120d8d48d664 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:53.552820 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c4aa7b87-711d-4214-9c3f-120d8d48d664 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:53.553299 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c4aa7b87-711d-4214-9c3f-120d8d48d664 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:53.553938 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c4aa7b87-711d-4214-9c3f-120d8d48d664 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:53.554641 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c4aa7b87-711d-4214-9c3f-120d8d48d664 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:53.555254 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c4aa7b87-711d-4214-9c3f-120d8d48d664 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:53.556331 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c4aa7b87-711d-4214-9c3f-120d8d48d664 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:53.710390 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c4aa7b87-711d-4214-9c3f-120d8d48d664 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:53.720211 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c4aa7b87-711d-4214-9c3f-120d8d48d664 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:53.721702 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c4aa7b87-711d-4214-9c3f-120d8d48d664 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:53.722951 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c4aa7b87-711d-4214-9c3f-120d8d48d664 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.181527 Oct 24 08:59:53.723715 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 433/859] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:53 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:53.782452 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4d7a5e89-46cf-437f-b015-7ffacd1a15f9 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:53.783088 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4d7a5e89-46cf-437f-b015-7ffacd1a15f9 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:53.791659 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d7a5e89-46cf-437f-b015-7ffacd1a15f9 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:53.792185 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d7a5e89-46cf-437f-b015-7ffacd1a15f9 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:53.793191 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d7a5e89-46cf-437f-b015-7ffacd1a15f9 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:53.793919 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d7a5e89-46cf-437f-b015-7ffacd1a15f9 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:53.794376 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d7a5e89-46cf-437f-b015-7ffacd1a15f9 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:53.794877 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4d7a5e89-46cf-437f-b015-7ffacd1a15f9 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:53.865187 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4d7a5e89-46cf-437f-b015-7ffacd1a15f9 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:53.878579 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4d7a5e89-46cf-437f-b015-7ffacd1a15f9 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1473 microversion: 2.1 time: 0.100340 Oct 24 08:59:53.880766 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 427/860] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:53 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1473 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:54.749950 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a4245b6b-6d68-4cad-8efc-7105e092ab8d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:54.750563 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a4245b6b-6d68-4cad-8efc-7105e092ab8d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:54.757561 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4245b6b-6d68-4cad-8efc-7105e092ab8d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:54.758448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4245b6b-6d68-4cad-8efc-7105e092ab8d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:54.758956 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4245b6b-6d68-4cad-8efc-7105e092ab8d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:54.759669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4245b6b-6d68-4cad-8efc-7105e092ab8d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:54.760358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4245b6b-6d68-4cad-8efc-7105e092ab8d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:54.760782 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a4245b6b-6d68-4cad-8efc-7105e092ab8d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:54.779020 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=2 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:54.904533 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-58171e80-d467-4240-9190-c68994df53d3 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:54.906135 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-58171e80-d467-4240-9190-c68994df53d3 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:54.916272 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-58171e80-d467-4240-9190-c68994df53d3 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:54.917134 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-58171e80-d467-4240-9190-c68994df53d3 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:54.917801 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-58171e80-d467-4240-9190-c68994df53d3 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:54.919939 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-58171e80-d467-4240-9190-c68994df53d3 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:54.921453 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-58171e80-d467-4240-9190-c68994df53d3 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:54.922120 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-58171e80-d467-4240-9190-c68994df53d3 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:54.954388 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a4245b6b-6d68-4cad-8efc-7105e092ab8d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:54.971268 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a4245b6b-6d68-4cad-8efc-7105e092ab8d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:54.973144 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a4245b6b-6d68-4cad-8efc-7105e092ab8d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:54.993631 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a4245b6b-6d68-4cad-8efc-7105e092ab8d tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.247929 Oct 24 08:59:54.994299 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 434/861] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:54 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:55.025402 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-58171e80-d467-4240-9190-c68994df53d3 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:55.029034 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:55.035275 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-58171e80-d467-4240-9190-c68994df53d3 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1473 microversion: 2.1 time: 0.136070 Oct 24 08:59:55.046143 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 428/862] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:54 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1473 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:59:56.027848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-00c7fcd7-9c75-46f9-9138-7ee89409d235 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:56.028541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-00c7fcd7-9c75-46f9-9138-7ee89409d235 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:56.036537 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-00c7fcd7-9c75-46f9-9138-7ee89409d235 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:56.037050 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-00c7fcd7-9c75-46f9-9138-7ee89409d235 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:56.037381 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-00c7fcd7-9c75-46f9-9138-7ee89409d235 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:56.038324 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-00c7fcd7-9c75-46f9-9138-7ee89409d235 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:56.038843 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-00c7fcd7-9c75-46f9-9138-7ee89409d235 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:56.039453 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-00c7fcd7-9c75-46f9-9138-7ee89409d235 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:56.059200 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7563e83a-bb6e-4100-be09-3d98872af7b7 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:56.059830 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7563e83a-bb6e-4100-be09-3d98872af7b7 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:56.065143 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=68,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:56.081777 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7563e83a-bb6e-4100-be09-3d98872af7b7 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:56.082296 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7563e83a-bb6e-4100-be09-3d98872af7b7 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:56.083030 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7563e83a-bb6e-4100-be09-3d98872af7b7 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:56.084303 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7563e83a-bb6e-4100-be09-3d98872af7b7 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:56.084965 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7563e83a-bb6e-4100-be09-3d98872af7b7 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:56.086666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7563e83a-bb6e-4100-be09-3d98872af7b7 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:56.206769 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7563e83a-bb6e-4100-be09-3d98872af7b7 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:56.217921 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7563e83a-bb6e-4100-be09-3d98872af7b7 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1473 microversion: 2.1 time: 0.162503 Oct 24 08:59:56.219167 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 429/863] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:56 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1473 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 08:59:56.246670 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-00c7fcd7-9c75-46f9-9138-7ee89409d235 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:56.256786 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-00c7fcd7-9c75-46f9-9138-7ee89409d235 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:56.258437 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-00c7fcd7-9c75-46f9-9138-7ee89409d235 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:56.261476 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-00c7fcd7-9c75-46f9-9138-7ee89409d235 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.240247 Oct 24 08:59:56.262073 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 435/864] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:56 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:57.245304 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ff018b4d-efe6-4ad5-ae5d-ae82b6ad7e41 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:57.251947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ff018b4d-efe6-4ad5-ae5d-ae82b6ad7e41 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:57.259491 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff018b4d-efe6-4ad5-ae5d-ae82b6ad7e41 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:57.259995 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff018b4d-efe6-4ad5-ae5d-ae82b6ad7e41 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:57.260502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff018b4d-efe6-4ad5-ae5d-ae82b6ad7e41 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:57.261227 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff018b4d-efe6-4ad5-ae5d-ae82b6ad7e41 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:57.262525 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff018b4d-efe6-4ad5-ae5d-ae82b6ad7e41 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:57.262525 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ff018b4d-efe6-4ad5-ae5d-ae82b6ad7e41 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:57.292352 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-413a1756-7fdb-4ac7-8dc3-c8625649448c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:57.293588 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-413a1756-7fdb-4ac7-8dc3-c8625649448c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:57.296937 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=65,nova_api:UPDATE=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:57.303652 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-413a1756-7fdb-4ac7-8dc3-c8625649448c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:57.304235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-413a1756-7fdb-4ac7-8dc3-c8625649448c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:57.304914 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-413a1756-7fdb-4ac7-8dc3-c8625649448c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:57.305843 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-413a1756-7fdb-4ac7-8dc3-c8625649448c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:57.314451 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-413a1756-7fdb-4ac7-8dc3-c8625649448c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:57.315091 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-413a1756-7fdb-4ac7-8dc3-c8625649448c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:57.350052 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ff018b4d-efe6-4ad5-ae5d-ae82b6ad7e41 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:57.359620 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ff018b4d-efe6-4ad5-ae5d-ae82b6ad7e41 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1304 microversion: 2.1 time: 0.120048 Oct 24 08:59:57.373647 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 430/865] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:57 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1304 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:57.670543 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-413a1756-7fdb-4ac7-8dc3-c8625649448c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:57.681585 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-413a1756-7fdb-4ac7-8dc3-c8625649448c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:57.684650 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-413a1756-7fdb-4ac7-8dc3-c8625649448c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:57.687472 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-413a1756-7fdb-4ac7-8dc3-c8625649448c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.399662 Oct 24 08:59:57.711111 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 436/866] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:57 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 08:59:58.391467 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-91b48d1e-313a-4004-a3af-f980690d8704 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:58.396098 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-91b48d1e-313a-4004-a3af-f980690d8704 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:58.406695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-91b48d1e-313a-4004-a3af-f980690d8704 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:58.407447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-91b48d1e-313a-4004-a3af-f980690d8704 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:58.408092 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-91b48d1e-313a-4004-a3af-f980690d8704 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:58.409169 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-91b48d1e-313a-4004-a3af-f980690d8704 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:58.409921 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-91b48d1e-313a-4004-a3af-f980690d8704 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:58.410512 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-91b48d1e-313a-4004-a3af-f980690d8704 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:58.513115 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-91b48d1e-313a-4004-a3af-f980690d8704 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:58.526985 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-91b48d1e-313a-4004-a3af-f980690d8704 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1304 microversion: 2.1 time: 0.143126 Oct 24 08:59:58.527801 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 431/867] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:58 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1304 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 08:59:58.724082 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-cb6a832c-e338-43e5-8d29-5366a41a5031 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:58.725334 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-cb6a832c-e338-43e5-8d29-5366a41a5031 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:58.732850 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cb6a832c-e338-43e5-8d29-5366a41a5031 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:58.733641 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cb6a832c-e338-43e5-8d29-5366a41a5031 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:58.734266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cb6a832c-e338-43e5-8d29-5366a41a5031 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:58.735265 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cb6a832c-e338-43e5-8d29-5366a41a5031 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:58.735918 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cb6a832c-e338-43e5-8d29-5366a41a5031 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:58.736587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cb6a832c-e338-43e5-8d29-5366a41a5031 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:59.125061 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cb6a832c-e338-43e5-8d29-5366a41a5031 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:59.135296 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cb6a832c-e338-43e5-8d29-5366a41a5031 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:59.139349 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cb6a832c-e338-43e5-8d29-5366a41a5031 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:59.142272 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-cb6a832c-e338-43e5-8d29-5366a41a5031 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.421938 Oct 24 08:59:59.163389 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 437/868] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 08:59:58 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 08:59:59.556378 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a6daed2e-ea86-4eb0-8b40-790d6092b75c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 08:59:59.557819 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-a6daed2e-ea86-4eb0-8b40-790d6092b75c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 08:59:59.564838 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a6daed2e-ea86-4eb0-8b40-790d6092b75c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:59.565684 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a6daed2e-ea86-4eb0-8b40-790d6092b75c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:59.566340 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a6daed2e-ea86-4eb0-8b40-790d6092b75c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:59.567531 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a6daed2e-ea86-4eb0-8b40-790d6092b75c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:59.568218 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a6daed2e-ea86-4eb0-8b40-790d6092b75c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:59.568881 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a6daed2e-ea86-4eb0-8b40-790d6092b75c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:59.619324 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a6daed2e-ea86-4eb0-8b40-790d6092b75c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 08:59:59.620265 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a6daed2e-ea86-4eb0-8b40-790d6092b75c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 08:59:59.620905 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a6daed2e-ea86-4eb0-8b40-790d6092b75c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 08:59:59.621588 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.objects.instance [None req-a6daed2e-ea86-4eb0-8b40-790d6092b75c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lazy-loading 'fault' on Instance uuid e7ae65b9-4038-4fad-b243-bbf157408181 {{(pid=130396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 08:59:59.684476 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-a6daed2e-ea86-4eb0-8b40-790d6092b75c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 08:59:59.720193 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-a6daed2e-ea86-4eb0-8b40-790d6092b75c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 200 len: 1309 microversion: 2.1 time: 0.168919 Oct 24 08:59:59.721505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 432/869] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 08:59:59 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 1309 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:00.173677 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d830903d-2f4a-4ede-a0f7-fec8a854ce1a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:00.174641 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d830903d-2f4a-4ede-a0f7-fec8a854ce1a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:00.182611 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d830903d-2f4a-4ede-a0f7-fec8a854ce1a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:00.183449 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d830903d-2f4a-4ede-a0f7-fec8a854ce1a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:00.183894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d830903d-2f4a-4ede-a0f7-fec8a854ce1a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:00.184653 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d830903d-2f4a-4ede-a0f7-fec8a854ce1a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:00.185310 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d830903d-2f4a-4ede-a0f7-fec8a854ce1a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:00.186198 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d830903d-2f4a-4ede-a0f7-fec8a854ce1a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:00.385195 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d830903d-2f4a-4ede-a0f7-fec8a854ce1a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:00.395072 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d830903d-2f4a-4ede-a0f7-fec8a854ce1a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:00.396786 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d830903d-2f4a-4ede-a0f7-fec8a854ce1a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:00.398078 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d830903d-2f4a-4ede-a0f7-fec8a854ce1a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.232023 Oct 24 09:00:00.398726 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 438/870] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:00:00 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 09:00:00.750362 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4bb69b75-92d8-4c87-80f3-aa30e749f3de tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:00.751598 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4bb69b75-92d8-4c87-80f3-aa30e749f3de tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: e7ae65b9-4038-4fad-b243-bbf157408181] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:00.760012 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4bb69b75-92d8-4c87-80f3-aa30e749f3de tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:00.760764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4bb69b75-92d8-4c87-80f3-aa30e749f3de tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:00.761259 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4bb69b75-92d8-4c87-80f3-aa30e749f3de tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:00.762200 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4bb69b75-92d8-4c87-80f3-aa30e749f3de tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:00.762874 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4bb69b75-92d8-4c87-80f3-aa30e749f3de tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:00.764857 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4bb69b75-92d8-4c87-80f3-aa30e749f3de tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:00.780109 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-4bb69b75-92d8-4c87-80f3-aa30e749f3de tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] HTTP exception thrown: Instance e7ae65b9-4038-4fad-b243-bbf157408181 could not be found. Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4bb69b75-92d8-4c87-80f3-aa30e749f3de tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Request method failure captured: Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 HTTP/1.1 Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlO1a4gxS_RX3u8K2x4yhOW9vrETflXSD23tAtVBS7hkMwBWoZeh-Y2ZlFaHsFODJyK_i6NcFeyOGykc4y3wOcVrsevFrHzoHRExS-f_cS-PyU4duotodp4i2nhk5ZvdtlDjgwPWraqFzelYlfiLV20uqQwKz_c-_qjQoZ7KagjfL3U Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 01318d53c6ed499e94a25e12f826719f Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-VolumesSnapshotTestJSON-68320019 Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 09:00:00.782170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 09:00:00.783636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/01318d53c6ed499e94a25e12f826719f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_01318d53c6ed499e94a25e12f826719f", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/01318d53c6ed499e94a25e12f826719f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/01318d53c6ed499e94a25e12f826719f"}]}] Oct 24 09:00:00.783636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-VolumesSnapshotTestJSON-68320019 Oct 24 09:00:00.783636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 01318d53c6ed499e94a25e12f826719f Oct 24 09:00:00.783636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-68320019 Oct 24 09:00:00.783636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-VolumesSnapshotTestJSON-68320019-project Oct 24 09:00:00.783636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 09:00:00.783636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 09:00:00.783636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: c94f6e2fa2b94630a745af7c54ec680a Oct 24 09:00:00.783636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-VolumesSnapshotTestJSON-68320019-project Oct 24 09:00:00.783636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 09:00:00.783636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance e7ae65b9-4038-4fad-b243-bbf157408181 could not be found. Oct 24 09:00:00.783636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': 'e7ae65b9-4038-4fad-b243-bbf157408181'} Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e7ae65b9-4038-4fad-b243-bbf157408181 could not be found. Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:00:00.784518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e7ae65b9-4038-4fad-b243-bbf157408181 could not be found. Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e7ae65b9-4038-4fad-b243-bbf157408181 could not be found. Oct 24 09:00:00.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:00:00.787930 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4bb69b75-92d8-4c87-80f3-aa30e749f3de tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Returning 404 to user: Instance e7ae65b9-4038-4fad-b243-bbf157408181 could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:00:00.790219 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4bb69b75-92d8-4c87-80f3-aa30e749f3de tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181" status: 404 len: 111 microversion: 2.1 time: 0.045304 Oct 24 09:00:00.792166 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 433/871] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:00 2022] GET /compute/v2.1/servers/e7ae65b9-4038-4fad-b243-bbf157408181 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:00:01.225455 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e15ba57c-0dc3-41f9-98cd-b4dc4f546f7a tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Action: 'create', calling method: .version_select at 0x7fdfedc0ad30>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-593019647", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBTpeE5h6OblGW8cEVGVfHopw4DqnckiDe8yINxVAANdlTvc1ZuGEOwn8v3n4uG71KtMtZkeA3DcZtamdrpX/0iDO635W+16KVfsU6F3o2V7oH7SWg0R1z5LtwrndoN4CA=="}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:00:01.242573 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-e15ba57c-0dc3-41f9-98cd-b4dc4f546f7a tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Getting quotas for project 30742e7b914243df8dd120079560a3de. Resources: {'key_pairs'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 09:00:01.248113 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-e15ba57c-0dc3-41f9-98cd-b4dc4f546f7a tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Getting quotas for user 3cddd27c37514e74a38430c46cef16fa and project 30742e7b914243df8dd120079560a3de. Resources: {'key_pairs'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 09:00:01.275283 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e15ba57c-0dc3-41f9-98cd-b4dc4f546f7a tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 399 microversion: 2.1 time: 0.281321 Oct 24 09:00:01.275891 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 439/872] 10.222.0.39 () {60 vars in 1159 bytes} [Mon Oct 24 09:00:00 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:00:01.425416 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-d20c2a51-900d-4c09-8f30-782ef8f9694a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:01.426183 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-d20c2a51-900d-4c09-8f30-782ef8f9694a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:01.432596 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d20c2a51-900d-4c09-8f30-782ef8f9694a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:01.433209 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d20c2a51-900d-4c09-8f30-782ef8f9694a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:01.437298 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d20c2a51-900d-4c09-8f30-782ef8f9694a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:01.438959 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d20c2a51-900d-4c09-8f30-782ef8f9694a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:01.439826 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d20c2a51-900d-4c09-8f30-782ef8f9694a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:01.440572 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d20c2a51-900d-4c09-8f30-782ef8f9694a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:01.633321 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d20c2a51-900d-4c09-8f30-782ef8f9694a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:01.643875 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d20c2a51-900d-4c09-8f30-782ef8f9694a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:01.645690 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d20c2a51-900d-4c09-8f30-782ef8f9694a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:01.646958 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-d20c2a51-900d-4c09-8f30-782ef8f9694a tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.225634 Oct 24 09:00:01.647613 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 434/873] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:00:01 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 09:00:02.674688 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1648ee53-f169-4836-8c54-1771435ab22c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:02.675336 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1648ee53-f169-4836-8c54-1771435ab22c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:02.683304 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1648ee53-f169-4836-8c54-1771435ab22c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:02.684035 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1648ee53-f169-4836-8c54-1771435ab22c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:02.684605 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1648ee53-f169-4836-8c54-1771435ab22c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:02.685371 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1648ee53-f169-4836-8c54-1771435ab22c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:02.685833 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1648ee53-f169-4836-8c54-1771435ab22c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:02.686301 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1648ee53-f169-4836-8c54-1771435ab22c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:02.832534 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1648ee53-f169-4836-8c54-1771435ab22c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:02.843697 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1648ee53-f169-4836-8c54-1771435ab22c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:02.845431 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1648ee53-f169-4836-8c54-1771435ab22c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:02.846907 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1648ee53-f169-4836-8c54-1771435ab22c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.176795 Oct 24 09:00:02.847642 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 440/874] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:00:02 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 09:00:03.874235 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0532f454-8436-4a53-8a3b-ff0673b40e4c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:03.874865 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-0532f454-8436-4a53-8a3b-ff0673b40e4c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:03.881035 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0532f454-8436-4a53-8a3b-ff0673b40e4c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:03.881492 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0532f454-8436-4a53-8a3b-ff0673b40e4c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:03.881923 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0532f454-8436-4a53-8a3b-ff0673b40e4c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:03.882621 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0532f454-8436-4a53-8a3b-ff0673b40e4c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:03.883067 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0532f454-8436-4a53-8a3b-ff0673b40e4c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:03.883494 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0532f454-8436-4a53-8a3b-ff0673b40e4c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:04.036455 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0532f454-8436-4a53-8a3b-ff0673b40e4c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:04.054934 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0532f454-8436-4a53-8a3b-ff0673b40e4c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:04.056750 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0532f454-8436-4a53-8a3b-ff0673b40e4c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:04.058024 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-0532f454-8436-4a53-8a3b-ff0673b40e4c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.188655 Oct 24 09:00:04.058546 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 435/875] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:00:03 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 09:00:05.088266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-bb9faee1-6d86-4cd7-806f-139db0d6cb70 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:05.089080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-bb9faee1-6d86-4cd7-806f-139db0d6cb70 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:05.099958 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bb9faee1-6d86-4cd7-806f-139db0d6cb70 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:05.100901 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bb9faee1-6d86-4cd7-806f-139db0d6cb70 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:05.101673 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bb9faee1-6d86-4cd7-806f-139db0d6cb70 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:05.102599 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bb9faee1-6d86-4cd7-806f-139db0d6cb70 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:05.103750 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bb9faee1-6d86-4cd7-806f-139db0d6cb70 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:05.104427 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bb9faee1-6d86-4cd7-806f-139db0d6cb70 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:05.255185 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-bb9faee1-6d86-4cd7-806f-139db0d6cb70 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:05.263327 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-bb9faee1-6d86-4cd7-806f-139db0d6cb70 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:05.264762 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-bb9faee1-6d86-4cd7-806f-139db0d6cb70 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:05.265915 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-bb9faee1-6d86-4cd7-806f-139db0d6cb70 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.182353 Oct 24 09:00:05.266471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 441/876] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:00:05 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 09:00:06.291283 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-18df90fb-0dc1-4de3-a330-73e67960d0df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:06.291897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-18df90fb-0dc1-4de3-a330-73e67960d0df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:06.300461 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-18df90fb-0dc1-4de3-a330-73e67960d0df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:06.300959 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-18df90fb-0dc1-4de3-a330-73e67960d0df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:06.301402 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-18df90fb-0dc1-4de3-a330-73e67960d0df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:06.302116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-18df90fb-0dc1-4de3-a330-73e67960d0df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:06.302566 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-18df90fb-0dc1-4de3-a330-73e67960d0df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:06.303063 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-18df90fb-0dc1-4de3-a330-73e67960d0df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:06.452269 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-18df90fb-0dc1-4de3-a330-73e67960d0df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:06.460353 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-18df90fb-0dc1-4de3-a330-73e67960d0df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:06.461786 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-18df90fb-0dc1-4de3-a330-73e67960d0df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:06.462934 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-18df90fb-0dc1-4de3-a330-73e67960d0df tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.175545 Oct 24 09:00:06.463479 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 436/877] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:00:06 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 09:00:07.291443 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-7a9300a9-52ec-4a8f-b76f-b9de3165ee9b req-751a696e-5ea6-4202-a011-4ee65b7e1668 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e72acc7a-decb-46c7-9c51-f731e2c413b3", "name": "network-vif-plugged", "status": "completed", "tag": "d0e79b5c-b764-405d-8c7a-d5032e3c72a8"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:00:07.302597 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-7a9300a9-52ec-4a8f-b76f-b9de3165ee9b req-751a696e-5ea6-4202-a011-4ee65b7e1668 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:07.303269 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-7a9300a9-52ec-4a8f-b76f-b9de3165ee9b req-751a696e-5ea6-4202-a011-4ee65b7e1668 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:07.303698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-7a9300a9-52ec-4a8f-b76f-b9de3165ee9b req-751a696e-5ea6-4202-a011-4ee65b7e1668 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:07.315473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-7a9300a9-52ec-4a8f-b76f-b9de3165ee9b req-751a696e-5ea6-4202-a011-4ee65b7e1668 service nova] Creating event network-vif-plugged:d0e79b5c-b764-405d-8c7a-d5032e3c72a8 for instance e72acc7a-decb-46c7-9c51-f731e2c413b3 on hv-cinder-maste Oct 24 09:00:07.320726 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-7a9300a9-52ec-4a8f-b76f-b9de3165ee9b req-751a696e-5ea6-4202-a011-4ee65b7e1668 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033513 Oct 24 09:00:07.321328 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 442/878] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:00:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 09:00:07.489922 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c0ef49a2-fb67-45e7-b4bc-47644489af67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:07.490702 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c0ef49a2-fb67-45e7-b4bc-47644489af67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:07.505577 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0ef49a2-fb67-45e7-b4bc-47644489af67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:07.506356 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0ef49a2-fb67-45e7-b4bc-47644489af67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:07.506794 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0ef49a2-fb67-45e7-b4bc-47644489af67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:07.507557 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0ef49a2-fb67-45e7-b4bc-47644489af67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:07.508072 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0ef49a2-fb67-45e7-b4bc-47644489af67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:07.508647 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c0ef49a2-fb67-45e7-b4bc-47644489af67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:07.685936 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c0ef49a2-fb67-45e7-b4bc-47644489af67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:07.699690 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c0ef49a2-fb67-45e7-b4bc-47644489af67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:07.702374 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c0ef49a2-fb67-45e7-b4bc-47644489af67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:07.703719 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c0ef49a2-fb67-45e7-b4bc-47644489af67 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.219218 Oct 24 09:00:07.704518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 437/879] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:00:07 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 09:00:08.731645 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ef0998bd-487d-4709-813f-c2936204a7fa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:08.732879 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ef0998bd-487d-4709-813f-c2936204a7fa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:08.738869 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef0998bd-487d-4709-813f-c2936204a7fa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:08.739356 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef0998bd-487d-4709-813f-c2936204a7fa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:08.739811 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef0998bd-487d-4709-813f-c2936204a7fa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:08.740768 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef0998bd-487d-4709-813f-c2936204a7fa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:08.741220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef0998bd-487d-4709-813f-c2936204a7fa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:08.741648 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef0998bd-487d-4709-813f-c2936204a7fa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:08.880396 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ef0998bd-487d-4709-813f-c2936204a7fa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:08.889739 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ef0998bd-487d-4709-813f-c2936204a7fa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:08.891178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ef0998bd-487d-4709-813f-c2936204a7fa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:08.892361 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ef0998bd-487d-4709-813f-c2936204a7fa tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.165760 Oct 24 09:00:08.892923 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 443/880] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:00:08 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 09:00:09.918343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-d37037fe-c15e-4f1e-aa38-d5c5eb217ef0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:09.918950 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-d37037fe-c15e-4f1e-aa38-d5c5eb217ef0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:09.925689 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d37037fe-c15e-4f1e-aa38-d5c5eb217ef0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:09.926183 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d37037fe-c15e-4f1e-aa38-d5c5eb217ef0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:09.926624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d37037fe-c15e-4f1e-aa38-d5c5eb217ef0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:09.927322 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d37037fe-c15e-4f1e-aa38-d5c5eb217ef0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:09.927780 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d37037fe-c15e-4f1e-aa38-d5c5eb217ef0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:09.928249 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d37037fe-c15e-4f1e-aa38-d5c5eb217ef0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:10.062693 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d37037fe-c15e-4f1e-aa38-d5c5eb217ef0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:10.070982 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d37037fe-c15e-4f1e-aa38-d5c5eb217ef0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:10.072828 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d37037fe-c15e-4f1e-aa38-d5c5eb217ef0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:10.074000 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-d37037fe-c15e-4f1e-aa38-d5c5eb217ef0 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.159611 Oct 24 09:00:10.074546 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 438/881] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:00:09 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 24 09:00:11.099594 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a125ad8f-fc69-442d-95ba-beaadb3892a3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:11.100331 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a125ad8f-fc69-442d-95ba-beaadb3892a3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:11.107162 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a125ad8f-fc69-442d-95ba-beaadb3892a3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:11.107654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a125ad8f-fc69-442d-95ba-beaadb3892a3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:11.108100 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a125ad8f-fc69-442d-95ba-beaadb3892a3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:11.108831 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a125ad8f-fc69-442d-95ba-beaadb3892a3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:11.109287 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a125ad8f-fc69-442d-95ba-beaadb3892a3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:11.109715 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a125ad8f-fc69-442d-95ba-beaadb3892a3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:11.267481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=4 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:00:11.398690 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a125ad8f-fc69-442d-95ba-beaadb3892a3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:11.407374 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a125ad8f-fc69-442d-95ba-beaadb3892a3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:11.408779 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a125ad8f-fc69-442d-95ba-beaadb3892a3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:11.409953 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a125ad8f-fc69-442d-95ba-beaadb3892a3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.314716 Oct 24 09:00:11.410486 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 444/882] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:00:11 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 09:00:12.435183 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8838ac40-f3e8-47f2-855c-c460d6ea1cd9 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:12.435804 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8838ac40-f3e8-47f2-855c-c460d6ea1cd9 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:12.442185 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8838ac40-f3e8-47f2-855c-c460d6ea1cd9 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:12.442669 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8838ac40-f3e8-47f2-855c-c460d6ea1cd9 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:12.443108 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8838ac40-f3e8-47f2-855c-c460d6ea1cd9 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:12.443802 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8838ac40-f3e8-47f2-855c-c460d6ea1cd9 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:12.444279 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8838ac40-f3e8-47f2-855c-c460d6ea1cd9 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:12.444703 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8838ac40-f3e8-47f2-855c-c460d6ea1cd9 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:12.579870 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8838ac40-f3e8-47f2-855c-c460d6ea1cd9 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:12.588024 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8838ac40-f3e8-47f2-855c-c460d6ea1cd9 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:12.589505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8838ac40-f3e8-47f2-855c-c460d6ea1cd9 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:12.590675 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8838ac40-f3e8-47f2-855c-c460d6ea1cd9 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.159624 Oct 24 09:00:12.591902 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 439/883] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:00:12 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 09:00:13.619851 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a217f1be-e6cc-4dbb-883d-c085c7218314 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:13.620681 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a217f1be-e6cc-4dbb-883d-c085c7218314 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:13.629406 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a217f1be-e6cc-4dbb-883d-c085c7218314 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:13.629893 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a217f1be-e6cc-4dbb-883d-c085c7218314 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:13.630326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a217f1be-e6cc-4dbb-883d-c085c7218314 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:13.631023 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a217f1be-e6cc-4dbb-883d-c085c7218314 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:13.631476 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a217f1be-e6cc-4dbb-883d-c085c7218314 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:13.631912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a217f1be-e6cc-4dbb-883d-c085c7218314 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:13.788902 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a217f1be-e6cc-4dbb-883d-c085c7218314 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:13.797503 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a217f1be-e6cc-4dbb-883d-c085c7218314 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:13.799030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a217f1be-e6cc-4dbb-883d-c085c7218314 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:13.801174 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a217f1be-e6cc-4dbb-883d-c085c7218314 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.186297 Oct 24 09:00:13.801174 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 445/884] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:00:13 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 09:00:14.827320 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-22185ca9-00ae-486d-8472-28c3ce9b1972 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:14.827873 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-22185ca9-00ae-486d-8472-28c3ce9b1972 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:14.834465 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-22185ca9-00ae-486d-8472-28c3ce9b1972 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:14.835046 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-22185ca9-00ae-486d-8472-28c3ce9b1972 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:14.835528 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-22185ca9-00ae-486d-8472-28c3ce9b1972 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:14.836303 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-22185ca9-00ae-486d-8472-28c3ce9b1972 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:14.836795 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-22185ca9-00ae-486d-8472-28c3ce9b1972 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:14.837227 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-22185ca9-00ae-486d-8472-28c3ce9b1972 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:14.988329 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-22185ca9-00ae-486d-8472-28c3ce9b1972 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:14.997880 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-22185ca9-00ae-486d-8472-28c3ce9b1972 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:14.999326 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-22185ca9-00ae-486d-8472-28c3ce9b1972 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:15.000536 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-22185ca9-00ae-486d-8472-28c3ce9b1972 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.178163 Oct 24 09:00:15.001108 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 440/885] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:00:14 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 09:00:15.057259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1723062069", "imageRef": "bba331ba-faf5-4a56-81d9-6e84e9609665", "flavorRef": "452", "networks": [{"uuid": "d6be167f-3870-4d45-a512-3d97a0f21398"}]}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:00:15.111760 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.network.neutron [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] validate_networks() for [('d6be167f-3870-4d45-a512-3d97a0f21398', None, None, None, None, None)] {{(pid=130395) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 09:00:15.740772 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Flavor limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 09:00:15.741199 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Image limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 09:00:15.741766 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Flavor pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 09:00:15.742312 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Image pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 09:00:15.742827 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 09:00:15.753394 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:15.753955 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:15.754457 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:15.755278 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:15.755738 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:15.756188 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:15.773377 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Getting quotas for project 01318d53c6ed499e94a25e12f826719f. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 09:00:15.777561 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Getting quotas for user c94f6e2fa2b94630a745af7c54ec680a and project 01318d53c6ed499e94a25e12f826719f. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 09:00:15.789759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Going to run 1 instances... {{(pid=130395) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 09:00:15.792059 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bba331ba-faf5-4a56-81d9-6e84e9609665',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130395) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 09:00:15.866446 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8abcc464-5dfe-40f4-9629-24d7c5eaeb4a tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.813002 Oct 24 09:00:15.867077 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 446/886] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 09:00:15 2022] POST /compute/v2.1/servers => generated 380 bytes in 814 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 09:00:15.878679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-12c10f17-6f4b-49bc-bb68-8677d8d2f260 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:15.879892 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-12c10f17-6f4b-49bc-bb68-8677d8d2f260 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:15.954883 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-12c10f17-6f4b-49bc-bb68-8677d8d2f260 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:15.969289 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-12c10f17-6f4b-49bc-bb68-8677d8d2f260 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 200 len: 1219 microversion: 2.1 time: 0.094493 Oct 24 09:00:15.970823 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 441/887] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:15 2022] GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 1219 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:16.026127 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-6c8cd6f5-cbb5-4866-8db8-8a9ae51e0ee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:16.026751 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-6c8cd6f5-cbb5-4866-8db8-8a9ae51e0ee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:16.034066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6c8cd6f5-cbb5-4866-8db8-8a9ae51e0ee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:16.035201 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6c8cd6f5-cbb5-4866-8db8-8a9ae51e0ee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:16.035647 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6c8cd6f5-cbb5-4866-8db8-8a9ae51e0ee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:16.036614 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6c8cd6f5-cbb5-4866-8db8-8a9ae51e0ee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:16.037076 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6c8cd6f5-cbb5-4866-8db8-8a9ae51e0ee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:16.037505 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6c8cd6f5-cbb5-4866-8db8-8a9ae51e0ee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:16.201940 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-6c8cd6f5-cbb5-4866-8db8-8a9ae51e0ee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:16.213146 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-6c8cd6f5-cbb5-4866-8db8-8a9ae51e0ee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:16.215364 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-6c8cd6f5-cbb5-4866-8db8-8a9ae51e0ee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:16.218213 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-6c8cd6f5-cbb5-4866-8db8-8a9ae51e0ee4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1730 microversion: 2.20 time: 0.196077 Oct 24 09:00:16.218978 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 447/888] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:00:16 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1730 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 09:00:16.993610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4ce6150d-f088-48fb-8a15-136a8fb32f79 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:16.995451 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4ce6150d-f088-48fb-8a15-136a8fb32f79 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:17.003970 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4ce6150d-f088-48fb-8a15-136a8fb32f79 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:17.004481 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4ce6150d-f088-48fb-8a15-136a8fb32f79 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:17.004918 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4ce6150d-f088-48fb-8a15-136a8fb32f79 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:17.005621 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4ce6150d-f088-48fb-8a15-136a8fb32f79 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:17.006071 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4ce6150d-f088-48fb-8a15-136a8fb32f79 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:17.006490 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4ce6150d-f088-48fb-8a15-136a8fb32f79 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:17.072633 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4ce6150d-f088-48fb-8a15-136a8fb32f79 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:17.081277 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4ce6150d-f088-48fb-8a15-136a8fb32f79 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 200 len: 1271 microversion: 2.1 time: 0.091698 Oct 24 09:00:17.081833 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 442/889] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:16 2022] GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 1271 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:17.244909 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-3ed858f6-64d3-4d28-b78c-42e131faa3e3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:17.246223 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-3ed858f6-64d3-4d28-b78c-42e131faa3e3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:17.253716 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ed858f6-64d3-4d28-b78c-42e131faa3e3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:17.254540 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ed858f6-64d3-4d28-b78c-42e131faa3e3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:17.255169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ed858f6-64d3-4d28-b78c-42e131faa3e3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:17.256025 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ed858f6-64d3-4d28-b78c-42e131faa3e3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:17.256714 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ed858f6-64d3-4d28-b78c-42e131faa3e3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:17.257366 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ed858f6-64d3-4d28-b78c-42e131faa3e3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:17.406622 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-3ed858f6-64d3-4d28-b78c-42e131faa3e3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:17.415335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-3ed858f6-64d3-4d28-b78c-42e131faa3e3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:17.416816 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-3ed858f6-64d3-4d28-b78c-42e131faa3e3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:17.417965 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-3ed858f6-64d3-4d28-b78c-42e131faa3e3 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1717 microversion: 2.20 time: 0.177038 Oct 24 09:00:17.418509 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 448/890] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:00:17 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1717 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 09:00:18.106011 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4bfd834f-6fe4-4714-94ff-cd9c25acff1b tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:18.107132 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4bfd834f-6fe4-4714-94ff-cd9c25acff1b tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:18.114104 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4bfd834f-6fe4-4714-94ff-cd9c25acff1b tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:18.114592 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4bfd834f-6fe4-4714-94ff-cd9c25acff1b tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:18.115033 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4bfd834f-6fe4-4714-94ff-cd9c25acff1b tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:18.115730 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4bfd834f-6fe4-4714-94ff-cd9c25acff1b tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:18.116211 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4bfd834f-6fe4-4714-94ff-cd9c25acff1b tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:18.116640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4bfd834f-6fe4-4714-94ff-cd9c25acff1b tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:18.256733 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4bfd834f-6fe4-4714-94ff-cd9c25acff1b tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:18.265723 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4bfd834f-6fe4-4714-94ff-cd9c25acff1b tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 200 len: 1319 microversion: 2.1 time: 0.163549 Oct 24 09:00:18.266572 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 443/891] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:18 2022] GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 1319 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:18.630855 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-de24b202-19a8-4674-83c9-0a7244d87171 req-01881b14-a885-49cd-a85b-690336f90cde service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "735ff4f8-a704-4d00-b8f0-3d13e88bb28c", "tag": "8b267f47-1042-4faa-870e-5e525d61f3d4"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:00:18.641021 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-de24b202-19a8-4674-83c9-0a7244d87171 req-01881b14-a885-49cd-a85b-690336f90cde service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:18.641499 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-de24b202-19a8-4674-83c9-0a7244d87171 req-01881b14-a885-49cd-a85b-690336f90cde service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:18.641925 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-de24b202-19a8-4674-83c9-0a7244d87171 req-01881b14-a885-49cd-a85b-690336f90cde service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:18.655014 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-de24b202-19a8-4674-83c9-0a7244d87171 req-01881b14-a885-49cd-a85b-690336f90cde service nova] Creating event network-changed:8b267f47-1042-4faa-870e-5e525d61f3d4 for instance 735ff4f8-a704-4d00-b8f0-3d13e88bb28c on hv-cinder-maste Oct 24 09:00:18.660701 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-de24b202-19a8-4674-83c9-0a7244d87171 req-01881b14-a885-49cd-a85b-690336f90cde service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033129 Oct 24 09:00:18.661707 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 449/892] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:00:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:00:19.288907 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4dad3d9f-4bc4-438e-be48-cf9baafcfb8d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:19.289602 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4dad3d9f-4bc4-438e-be48-cf9baafcfb8d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:19.297354 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4dad3d9f-4bc4-438e-be48-cf9baafcfb8d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:19.297838 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4dad3d9f-4bc4-438e-be48-cf9baafcfb8d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:19.298265 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4dad3d9f-4bc4-438e-be48-cf9baafcfb8d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:19.298956 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4dad3d9f-4bc4-438e-be48-cf9baafcfb8d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:19.299401 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4dad3d9f-4bc4-438e-be48-cf9baafcfb8d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:19.299825 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4dad3d9f-4bc4-438e-be48-cf9baafcfb8d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:19.438022 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4dad3d9f-4bc4-438e-be48-cf9baafcfb8d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:19.445741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4dad3d9f-4bc4-438e-be48-cf9baafcfb8d tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 200 len: 1319 microversion: 2.1 time: 0.160679 Oct 24 09:00:19.446443 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 444/893] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:19 2022] GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 1319 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:20.471009 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-0903e228-216d-4c0b-b7c2-10ccb149df8c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:20.471814 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-0903e228-216d-4c0b-b7c2-10ccb149df8c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:20.478417 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0903e228-216d-4c0b-b7c2-10ccb149df8c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:20.478896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0903e228-216d-4c0b-b7c2-10ccb149df8c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:20.479331 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0903e228-216d-4c0b-b7c2-10ccb149df8c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:20.480021 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0903e228-216d-4c0b-b7c2-10ccb149df8c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:20.480501 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0903e228-216d-4c0b-b7c2-10ccb149df8c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:20.480924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-0903e228-216d-4c0b-b7c2-10ccb149df8c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:20.616210 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-0903e228-216d-4c0b-b7c2-10ccb149df8c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:20.624100 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-0903e228-216d-4c0b-b7c2-10ccb149df8c tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 200 len: 1319 microversion: 2.1 time: 0.157371 Oct 24 09:00:20.624676 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 450/894] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:20 2022] GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 1319 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:21.647569 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7ae49873-3984-44b9-b876-3acdd3071f25 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:21.648332 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7ae49873-3984-44b9-b876-3acdd3071f25 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:21.654851 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7ae49873-3984-44b9-b876-3acdd3071f25 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:21.655332 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7ae49873-3984-44b9-b876-3acdd3071f25 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:21.655764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7ae49873-3984-44b9-b876-3acdd3071f25 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:21.656482 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7ae49873-3984-44b9-b876-3acdd3071f25 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:21.656933 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7ae49873-3984-44b9-b876-3acdd3071f25 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:21.657354 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7ae49873-3984-44b9-b876-3acdd3071f25 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:21.797930 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7ae49873-3984-44b9-b876-3acdd3071f25 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:21.805483 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7ae49873-3984-44b9-b876-3acdd3071f25 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 200 len: 1319 microversion: 2.1 time: 0.161793 Oct 24 09:00:21.806047 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 445/895] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:21 2022] GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 1319 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:22.829156 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ffce18d7-1d35-4125-a5d4-10ce218cbcfc tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:22.829772 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ffce18d7-1d35-4125-a5d4-10ce218cbcfc tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:22.836174 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ffce18d7-1d35-4125-a5d4-10ce218cbcfc tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:22.836663 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ffce18d7-1d35-4125-a5d4-10ce218cbcfc tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:22.837100 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ffce18d7-1d35-4125-a5d4-10ce218cbcfc tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:22.837795 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ffce18d7-1d35-4125-a5d4-10ce218cbcfc tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:22.838250 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ffce18d7-1d35-4125-a5d4-10ce218cbcfc tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:22.838671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ffce18d7-1d35-4125-a5d4-10ce218cbcfc tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:22.968303 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ffce18d7-1d35-4125-a5d4-10ce218cbcfc tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:22.976769 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ffce18d7-1d35-4125-a5d4-10ce218cbcfc tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 200 len: 1319 microversion: 2.1 time: 0.152166 Oct 24 09:00:22.978734 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 451/896] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:22 2022] GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 1319 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:24.000321 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c70ae76a-fcf1-473a-930b-c81e0ab86202 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:24.001711 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c70ae76a-fcf1-473a-930b-c81e0ab86202 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:24.009698 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c70ae76a-fcf1-473a-930b-c81e0ab86202 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:24.010558 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c70ae76a-fcf1-473a-930b-c81e0ab86202 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:24.011298 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c70ae76a-fcf1-473a-930b-c81e0ab86202 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:24.013061 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c70ae76a-fcf1-473a-930b-c81e0ab86202 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:24.013787 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c70ae76a-fcf1-473a-930b-c81e0ab86202 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:24.014559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c70ae76a-fcf1-473a-930b-c81e0ab86202 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:24.186301 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c70ae76a-fcf1-473a-930b-c81e0ab86202 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:24.190063 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=100 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:00:24.195192 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c70ae76a-fcf1-473a-930b-c81e0ab86202 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 200 len: 1319 microversion: 2.1 time: 0.200072 Oct 24 09:00:24.195826 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 446/897] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:23 2022] GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 1319 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:25.218818 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1ce39be6-8142-4b7b-88a6-552168222838 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:25.219435 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1ce39be6-8142-4b7b-88a6-552168222838 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:25.226002 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ce39be6-8142-4b7b-88a6-552168222838 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:25.226489 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ce39be6-8142-4b7b-88a6-552168222838 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:25.226933 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ce39be6-8142-4b7b-88a6-552168222838 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:25.227630 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ce39be6-8142-4b7b-88a6-552168222838 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:25.228085 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ce39be6-8142-4b7b-88a6-552168222838 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:25.228542 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ce39be6-8142-4b7b-88a6-552168222838 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:25.231178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=88 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:00:25.360899 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1ce39be6-8142-4b7b-88a6-552168222838 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:25.369057 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1ce39be6-8142-4b7b-88a6-552168222838 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 200 len: 1319 microversion: 2.1 time: 0.154730 Oct 24 09:00:25.369618 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 452/898] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:25 2022] GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 1319 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:25.670395 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-6fa251be-5ddf-4eda-ae96-adf5897334af req-99541aae-1326-460a-9ba8-3a3e28556488 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "735ff4f8-a704-4d00-b8f0-3d13e88bb28c", "name": "network-vif-plugged", "status": "completed", "tag": "8b267f47-1042-4faa-870e-5e525d61f3d4"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:00:25.680725 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-6fa251be-5ddf-4eda-ae96-adf5897334af req-99541aae-1326-460a-9ba8-3a3e28556488 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:25.681213 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-6fa251be-5ddf-4eda-ae96-adf5897334af req-99541aae-1326-460a-9ba8-3a3e28556488 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:25.681665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-6fa251be-5ddf-4eda-ae96-adf5897334af req-99541aae-1326-460a-9ba8-3a3e28556488 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:25.693365 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-6fa251be-5ddf-4eda-ae96-adf5897334af req-99541aae-1326-460a-9ba8-3a3e28556488 service nova] Creating event network-vif-plugged:8b267f47-1042-4faa-870e-5e525d61f3d4 for instance 735ff4f8-a704-4d00-b8f0-3d13e88bb28c on hv-cinder-maste Oct 24 09:00:25.698356 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-6fa251be-5ddf-4eda-ae96-adf5897334af req-99541aae-1326-460a-9ba8-3a3e28556488 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032539 Oct 24 09:00:25.698950 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 447/899] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:00:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:00:26.394900 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e166dd30-01b2-4727-8f3a-ffa8faad8e2f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:26.395493 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e166dd30-01b2-4727-8f3a-ffa8faad8e2f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:26.397288 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=2 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:00:26.397943 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=6 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:00:26.404932 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e166dd30-01b2-4727-8f3a-ffa8faad8e2f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:26.405463 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e166dd30-01b2-4727-8f3a-ffa8faad8e2f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:26.405901 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e166dd30-01b2-4727-8f3a-ffa8faad8e2f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:26.406599 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e166dd30-01b2-4727-8f3a-ffa8faad8e2f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:26.407048 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e166dd30-01b2-4727-8f3a-ffa8faad8e2f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:26.407469 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e166dd30-01b2-4727-8f3a-ffa8faad8e2f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:26.538214 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e166dd30-01b2-4727-8f3a-ffa8faad8e2f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:26.545968 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e166dd30-01b2-4727-8f3a-ffa8faad8e2f tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 200 len: 1319 microversion: 2.1 time: 0.154959 Oct 24 09:00:26.546526 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 453/900] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:26 2022] GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 1319 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:27.565548 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=2 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:00:27.570359 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-895f05ec-a1e8-4023-8674-8a27d929f5bb tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:27.571056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-895f05ec-a1e8-4023-8674-8a27d929f5bb tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:27.572855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=42 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:00:27.579018 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-895f05ec-a1e8-4023-8674-8a27d929f5bb tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:27.579496 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-895f05ec-a1e8-4023-8674-8a27d929f5bb tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:27.579925 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-895f05ec-a1e8-4023-8674-8a27d929f5bb tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:27.580638 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-895f05ec-a1e8-4023-8674-8a27d929f5bb tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:27.581090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-895f05ec-a1e8-4023-8674-8a27d929f5bb tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:27.581510 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-895f05ec-a1e8-4023-8674-8a27d929f5bb tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:27.720912 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-895f05ec-a1e8-4023-8674-8a27d929f5bb tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:27.728624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-895f05ec-a1e8-4023-8674-8a27d929f5bb tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 200 len: 1319 microversion: 2.1 time: 0.164011 Oct 24 09:00:27.729199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 448/901] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:27 2022] GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 1319 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:28.753013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ad1e491d-c366-46d5-a462-ba87b211aa94 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:28.753693 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ad1e491d-c366-46d5-a462-ba87b211aa94 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:28.755913 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=56,nova_api:INSERT=4 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:00:28.761294 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ad1e491d-c366-46d5-a462-ba87b211aa94 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:28.761802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ad1e491d-c366-46d5-a462-ba87b211aa94 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:28.762264 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ad1e491d-c366-46d5-a462-ba87b211aa94 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:28.762967 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ad1e491d-c366-46d5-a462-ba87b211aa94 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:28.763428 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ad1e491d-c366-46d5-a462-ba87b211aa94 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:28.763858 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ad1e491d-c366-46d5-a462-ba87b211aa94 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:28.895196 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ad1e491d-c366-46d5-a462-ba87b211aa94 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:28.903992 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ad1e491d-c366-46d5-a462-ba87b211aa94 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 200 len: 1508 microversion: 2.1 time: 0.153931 Oct 24 09:00:28.904683 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 454/902] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:28 2022] GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 1508 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 09:00:28.926632 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4cdb4338-0ef0-4ff4-bf9d-c90e1b9b8e60 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4f402f7d-ed05-425d-9e70-ad22d5f86615", "device": "/dev/sdb"}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:00:28.929077 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4cdb4338-0ef0-4ff4-bf9d-c90e1b9b8e60 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:28.935934 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cdb4338-0ef0-4ff4-bf9d-c90e1b9b8e60 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:28.936456 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cdb4338-0ef0-4ff4-bf9d-c90e1b9b8e60 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:28.936897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cdb4338-0ef0-4ff4-bf9d-c90e1b9b8e60 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:28.937602 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cdb4338-0ef0-4ff4-bf9d-c90e1b9b8e60 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:28.938060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cdb4338-0ef0-4ff4-bf9d-c90e1b9b8e60 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:28.938492 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cdb4338-0ef0-4ff4-bf9d-c90e1b9b8e60 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:29.209015 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4cdb4338-0ef0-4ff4-bf9d-c90e1b9b8e60 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "POST /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.286170 Oct 24 09:00:29.209657 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 449/903] 10.222.0.39 () {60 vars in 1323 bytes} [Mon Oct 24 09:00:28 2022] POST /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c/os-volume_attachments => generated 194 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:00:31.596892 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-2143540443", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-593019647", "security_groups": [{"name": "tempest-secgroup-smoke-1512102953"}], "block_device_mapping_v2": [{"uuid": "c0807ccc-7846-4c0d-8cac-7c6f691a0ba9", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "15551789-d406-4f91-85fe-9eee40e4ab51"}]}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:00:31.710664 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.network.neutron [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] validate_networks() for [('15551789-d406-4f91-85fe-9eee40e4ab51', None, None, None, None, None)] {{(pid=130395) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 09:00:32.097337 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Flavor limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 09:00:32.097758 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Image limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 09:00:32.098224 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Flavor pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 09:00:32.098626 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Image pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 09:00:32.099032 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 09:00:32.109142 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:32.109617 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:32.110046 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:32.110761 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:32.111207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:32.111657 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:32.129206 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Getting quotas for project 30742e7b914243df8dd120079560a3de. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 09:00:32.133095 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Getting quotas for user 3cddd27c37514e74a38430c46cef16fa and project 30742e7b914243df8dd120079560a3de. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 09:00:32.194097 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Going to run 1 instances... {{(pid=130395) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 09:00:32.196355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='c0807ccc-7846-4c0d-8cac-7c6f691a0ba9',volume_size=None,volume_type=None)] {{(pid=130395) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 09:00:32.328257 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-803e960c-bcd7-4ac5-bf79-38c17e1ce020 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.735142 Oct 24 09:00:32.328899 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 455/904] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 09:00:31 2022] POST /compute/v2.1/servers => generated 406 bytes in 736 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 09:00:32.340967 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c04a3dca-36eb-4b16-a54f-abe3f0edad47 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:32.341744 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c04a3dca-36eb-4b16-a54f-abe3f0edad47 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:32.404069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c04a3dca-36eb-4b16-a54f-abe3f0edad47 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:32.411312 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c04a3dca-36eb-4b16-a54f-abe3f0edad47 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1094 microversion: 2.1 time: 0.075073 Oct 24 09:00:32.412121 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 450/905] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:32 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1094 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:33.435338 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-6d21c5c1-7dfb-45af-9dea-f66108d99502 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:33.435929 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-6d21c5c1-7dfb-45af-9dea-f66108d99502 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:33.442254 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d21c5c1-7dfb-45af-9dea-f66108d99502 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:33.442742 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d21c5c1-7dfb-45af-9dea-f66108d99502 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:33.443185 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d21c5c1-7dfb-45af-9dea-f66108d99502 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:33.443883 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d21c5c1-7dfb-45af-9dea-f66108d99502 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:33.444405 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d21c5c1-7dfb-45af-9dea-f66108d99502 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:33.444870 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6d21c5c1-7dfb-45af-9dea-f66108d99502 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:33.513444 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-6d21c5c1-7dfb-45af-9dea-f66108d99502 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:33.521617 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-6d21c5c1-7dfb-45af-9dea-f66108d99502 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1192 microversion: 2.1 time: 0.090882 Oct 24 09:00:33.522397 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 456/906] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:33 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1192 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:34.545664 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2afa1cf7-5a9d-4ca2-bbad-3c497b014d27 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:34.546470 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2afa1cf7-5a9d-4ca2-bbad-3c497b014d27 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:34.554317 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2afa1cf7-5a9d-4ca2-bbad-3c497b014d27 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:34.554932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2afa1cf7-5a9d-4ca2-bbad-3c497b014d27 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:34.555457 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2afa1cf7-5a9d-4ca2-bbad-3c497b014d27 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:34.556302 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2afa1cf7-5a9d-4ca2-bbad-3c497b014d27 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:34.556863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2afa1cf7-5a9d-4ca2-bbad-3c497b014d27 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:34.557403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2afa1cf7-5a9d-4ca2-bbad-3c497b014d27 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:34.696523 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2afa1cf7-5a9d-4ca2-bbad-3c497b014d27 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:34.705135 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2afa1cf7-5a9d-4ca2-bbad-3c497b014d27 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1278 microversion: 2.1 time: 0.163503 Oct 24 09:00:34.705698 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 451/907] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:34 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1278 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:35.060395 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-f5edb5cd-f677-4523-9358-aa52afcf5047 req-7d3028f1-247c-4db9-919b-f8c8132996cd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f2c4ad60-dd40-42b5-8728-8a414e8d7644", "tag": "ea79d810-db9b-4fba-9d68-f8b0560784ee"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:00:35.069795 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-f5edb5cd-f677-4523-9358-aa52afcf5047 req-7d3028f1-247c-4db9-919b-f8c8132996cd service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:35.070300 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-f5edb5cd-f677-4523-9358-aa52afcf5047 req-7d3028f1-247c-4db9-919b-f8c8132996cd service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:35.070731 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-f5edb5cd-f677-4523-9358-aa52afcf5047 req-7d3028f1-247c-4db9-919b-f8c8132996cd service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:35.081969 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-f5edb5cd-f677-4523-9358-aa52afcf5047 req-7d3028f1-247c-4db9-919b-f8c8132996cd service nova] Creating event network-changed:ea79d810-db9b-4fba-9d68-f8b0560784ee for instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 on hv-cinder-maste Oct 24 09:00:35.086921 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-f5edb5cd-f677-4523-9358-aa52afcf5047 req-7d3028f1-247c-4db9-919b-f8c8132996cd service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030462 Oct 24 09:00:35.087441 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 457/908] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:00:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:00:35.728582 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f383e46f-93fc-4adc-80cf-bf3688646f06 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:35.729249 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f383e46f-93fc-4adc-80cf-bf3688646f06 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:35.736774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f383e46f-93fc-4adc-80cf-bf3688646f06 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:35.739675 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f383e46f-93fc-4adc-80cf-bf3688646f06 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:35.740311 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f383e46f-93fc-4adc-80cf-bf3688646f06 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:35.740892 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f383e46f-93fc-4adc-80cf-bf3688646f06 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:35.741351 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f383e46f-93fc-4adc-80cf-bf3688646f06 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:35.741772 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f383e46f-93fc-4adc-80cf-bf3688646f06 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:35.895640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f383e46f-93fc-4adc-80cf-bf3688646f06 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:35.904427 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f383e46f-93fc-4adc-80cf-bf3688646f06 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1278 microversion: 2.1 time: 0.179665 Oct 24 09:00:35.905090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 452/909] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:35 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1278 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:36.929422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-bfbf2111-65c8-486d-b0fe-3e2e2990f128 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:36.930197 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-bfbf2111-65c8-486d-b0fe-3e2e2990f128 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:36.937106 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bfbf2111-65c8-486d-b0fe-3e2e2990f128 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:36.937597 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bfbf2111-65c8-486d-b0fe-3e2e2990f128 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:36.938037 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bfbf2111-65c8-486d-b0fe-3e2e2990f128 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:36.938747 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bfbf2111-65c8-486d-b0fe-3e2e2990f128 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:36.939204 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bfbf2111-65c8-486d-b0fe-3e2e2990f128 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:36.939802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bfbf2111-65c8-486d-b0fe-3e2e2990f128 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:37.082345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-bfbf2111-65c8-486d-b0fe-3e2e2990f128 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:37.091674 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-bfbf2111-65c8-486d-b0fe-3e2e2990f128 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1278 microversion: 2.1 time: 0.166872 Oct 24 09:00:37.092299 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 458/910] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:36 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1278 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:38.118026 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-d34452d2-e8f4-484c-8da6-814816a4fa4e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:38.118631 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-d34452d2-e8f4-484c-8da6-814816a4fa4e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:38.125693 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d34452d2-e8f4-484c-8da6-814816a4fa4e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:38.126169 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d34452d2-e8f4-484c-8da6-814816a4fa4e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:38.126600 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d34452d2-e8f4-484c-8da6-814816a4fa4e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:38.127283 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d34452d2-e8f4-484c-8da6-814816a4fa4e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:38.127730 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d34452d2-e8f4-484c-8da6-814816a4fa4e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:38.128157 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d34452d2-e8f4-484c-8da6-814816a4fa4e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:38.260374 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d34452d2-e8f4-484c-8da6-814816a4fa4e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:38.268186 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-d34452d2-e8f4-484c-8da6-814816a4fa4e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1266 microversion: 2.1 time: 0.154087 Oct 24 09:00:38.268755 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 453/911] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:38 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1266 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:39.290958 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a5be51c4-5a0c-4690-b344-403b5f6a1c13 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:39.291502 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a5be51c4-5a0c-4690-b344-403b5f6a1c13 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:39.298007 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5be51c4-5a0c-4690-b344-403b5f6a1c13 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:39.298493 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5be51c4-5a0c-4690-b344-403b5f6a1c13 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:39.298938 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5be51c4-5a0c-4690-b344-403b5f6a1c13 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:39.299634 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5be51c4-5a0c-4690-b344-403b5f6a1c13 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:39.300089 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5be51c4-5a0c-4690-b344-403b5f6a1c13 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:39.300561 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a5be51c4-5a0c-4690-b344-403b5f6a1c13 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:39.436461 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a5be51c4-5a0c-4690-b344-403b5f6a1c13 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:39.444285 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a5be51c4-5a0c-4690-b344-403b5f6a1c13 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1266 microversion: 2.1 time: 0.157248 Oct 24 09:00:39.444951 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 459/912] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:39 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1266 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:40.466687 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6d768195-543d-4315-8ece-2ee13fee2d74 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:40.467287 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6d768195-543d-4315-8ece-2ee13fee2d74 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:40.474028 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d768195-543d-4315-8ece-2ee13fee2d74 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:40.474524 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d768195-543d-4315-8ece-2ee13fee2d74 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:40.474973 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d768195-543d-4315-8ece-2ee13fee2d74 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:40.475678 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d768195-543d-4315-8ece-2ee13fee2d74 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:40.476154 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d768195-543d-4315-8ece-2ee13fee2d74 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:40.476590 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d768195-543d-4315-8ece-2ee13fee2d74 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:40.606034 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6d768195-543d-4315-8ece-2ee13fee2d74 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:40.614049 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6d768195-543d-4315-8ece-2ee13fee2d74 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1266 microversion: 2.1 time: 0.151215 Oct 24 09:00:40.614618 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 454/913] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:40 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1266 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:41.637571 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-4a6cd6f2-9510-4bba-a658-47ec5fa106ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:41.638178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-4a6cd6f2-9510-4bba-a658-47ec5fa106ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:41.644667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4a6cd6f2-9510-4bba-a658-47ec5fa106ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:41.645152 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4a6cd6f2-9510-4bba-a658-47ec5fa106ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:41.645592 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4a6cd6f2-9510-4bba-a658-47ec5fa106ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:41.646288 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4a6cd6f2-9510-4bba-a658-47ec5fa106ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:41.646743 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4a6cd6f2-9510-4bba-a658-47ec5fa106ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:41.647167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4a6cd6f2-9510-4bba-a658-47ec5fa106ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:41.782508 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-4a6cd6f2-9510-4bba-a658-47ec5fa106ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:41.790137 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-4a6cd6f2-9510-4bba-a658-47ec5fa106ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1266 microversion: 2.1 time: 0.156563 Oct 24 09:00:41.790706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 460/914] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:41 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1266 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:42.813839 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4cb4081d-94b9-4c1b-89e9-503a8fa36fda tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:42.814445 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4cb4081d-94b9-4c1b-89e9-503a8fa36fda tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:42.816925 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=2 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:00:42.822233 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cb4081d-94b9-4c1b-89e9-503a8fa36fda tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:42.822720 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cb4081d-94b9-4c1b-89e9-503a8fa36fda tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:42.823155 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cb4081d-94b9-4c1b-89e9-503a8fa36fda tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:42.823855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cb4081d-94b9-4c1b-89e9-503a8fa36fda tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:42.824401 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cb4081d-94b9-4c1b-89e9-503a8fa36fda tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:42.824866 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4cb4081d-94b9-4c1b-89e9-503a8fa36fda tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:42.956571 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4cb4081d-94b9-4c1b-89e9-503a8fa36fda tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:42.967260 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4cb4081d-94b9-4c1b-89e9-503a8fa36fda tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1266 microversion: 2.1 time: 0.157261 Oct 24 09:00:42.967852 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 455/915] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:42 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1266 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:43.991190 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-129b2426-b9cd-45cd-9234-476721ec1a8c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:43.992475 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-129b2426-b9cd-45cd-9234-476721ec1a8c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:43.994416 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=2 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:00:43.995088 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=6 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:00:44.001423 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-129b2426-b9cd-45cd-9234-476721ec1a8c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:44.001915 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-129b2426-b9cd-45cd-9234-476721ec1a8c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:44.002361 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-129b2426-b9cd-45cd-9234-476721ec1a8c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:44.003071 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-129b2426-b9cd-45cd-9234-476721ec1a8c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:44.003528 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-129b2426-b9cd-45cd-9234-476721ec1a8c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:44.003955 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-129b2426-b9cd-45cd-9234-476721ec1a8c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:44.145661 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-129b2426-b9cd-45cd-9234-476721ec1a8c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:44.153364 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-129b2426-b9cd-45cd-9234-476721ec1a8c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1266 microversion: 2.1 time: 0.166373 Oct 24 09:00:44.153940 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 461/916] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:43 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1266 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:45.176574 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-438c57cd-0d43-48d5-bb42-4606529cf879 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:45.177219 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-438c57cd-0d43-48d5-bb42-4606529cf879 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:45.184441 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-438c57cd-0d43-48d5-bb42-4606529cf879 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:45.184931 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-438c57cd-0d43-48d5-bb42-4606529cf879 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:45.185372 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-438c57cd-0d43-48d5-bb42-4606529cf879 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:45.186076 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-438c57cd-0d43-48d5-bb42-4606529cf879 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:45.186532 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-438c57cd-0d43-48d5-bb42-4606529cf879 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:45.186962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-438c57cd-0d43-48d5-bb42-4606529cf879 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:45.325381 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-438c57cd-0d43-48d5-bb42-4606529cf879 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:45.333054 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-438c57cd-0d43-48d5-bb42-4606529cf879 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1266 microversion: 2.1 time: 0.160739 Oct 24 09:00:45.333640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 456/917] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:45 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1266 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:46.355014 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c5350aff-83cb-4bc6-9a98-f081ac914def tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:46.356233 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c5350aff-83cb-4bc6-9a98-f081ac914def tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:46.362723 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c5350aff-83cb-4bc6-9a98-f081ac914def tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:46.363202 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c5350aff-83cb-4bc6-9a98-f081ac914def tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:46.364112 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c5350aff-83cb-4bc6-9a98-f081ac914def tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:46.364831 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c5350aff-83cb-4bc6-9a98-f081ac914def tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:46.365281 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c5350aff-83cb-4bc6-9a98-f081ac914def tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:46.365703 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c5350aff-83cb-4bc6-9a98-f081ac914def tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:46.497648 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c5350aff-83cb-4bc6-9a98-f081ac914def tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:46.505389 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c5350aff-83cb-4bc6-9a98-f081ac914def tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1266 microversion: 2.1 time: 0.154451 Oct 24 09:00:46.505952 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 462/918] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:46 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1266 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:47.530411 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-3c619042-5c9e-4217-abbc-aeb4d10bec87 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:47.531008 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-3c619042-5c9e-4217-abbc-aeb4d10bec87 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:47.537401 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3c619042-5c9e-4217-abbc-aeb4d10bec87 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:47.537876 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3c619042-5c9e-4217-abbc-aeb4d10bec87 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:47.538305 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3c619042-5c9e-4217-abbc-aeb4d10bec87 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:47.538986 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3c619042-5c9e-4217-abbc-aeb4d10bec87 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:47.539434 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3c619042-5c9e-4217-abbc-aeb4d10bec87 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:47.539850 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3c619042-5c9e-4217-abbc-aeb4d10bec87 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:47.676059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-3c619042-5c9e-4217-abbc-aeb4d10bec87 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:47.683760 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-3c619042-5c9e-4217-abbc-aeb4d10bec87 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1266 microversion: 2.1 time: 0.158318 Oct 24 09:00:47.684357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 457/919] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:47 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1266 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:48.707053 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-bd09a3ae-5cb8-48bc-aa7c-2b4f52594807 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:48.707624 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-bd09a3ae-5cb8-48bc-aa7c-2b4f52594807 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:48.713979 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bd09a3ae-5cb8-48bc-aa7c-2b4f52594807 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:48.714684 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bd09a3ae-5cb8-48bc-aa7c-2b4f52594807 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:48.715153 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bd09a3ae-5cb8-48bc-aa7c-2b4f52594807 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:48.715858 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bd09a3ae-5cb8-48bc-aa7c-2b4f52594807 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:48.716362 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bd09a3ae-5cb8-48bc-aa7c-2b4f52594807 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:48.716794 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bd09a3ae-5cb8-48bc-aa7c-2b4f52594807 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:48.846468 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-bd09a3ae-5cb8-48bc-aa7c-2b4f52594807 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:48.854239 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-bd09a3ae-5cb8-48bc-aa7c-2b4f52594807 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1266 microversion: 2.1 time: 0.151146 Oct 24 09:00:48.854793 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 463/920] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:48 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1266 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:49.878162 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f095af76-20cb-45c9-b66d-bdbd67e538c8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:49.878780 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f095af76-20cb-45c9-b66d-bdbd67e538c8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:49.885837 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f095af76-20cb-45c9-b66d-bdbd67e538c8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:49.886831 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f095af76-20cb-45c9-b66d-bdbd67e538c8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:49.887348 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f095af76-20cb-45c9-b66d-bdbd67e538c8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:49.888062 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f095af76-20cb-45c9-b66d-bdbd67e538c8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:49.888589 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f095af76-20cb-45c9-b66d-bdbd67e538c8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:49.889017 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f095af76-20cb-45c9-b66d-bdbd67e538c8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:50.026113 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f095af76-20cb-45c9-b66d-bdbd67e538c8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:50.034543 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f095af76-20cb-45c9-b66d-bdbd67e538c8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1266 microversion: 2.1 time: 0.160444 Oct 24 09:00:50.035226 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 458/921] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:49 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1266 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:51.056797 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-3c62683c-32bd-49ab-b58b-d14255763990 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:51.057405 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-3c62683c-32bd-49ab-b58b-d14255763990 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:51.064415 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3c62683c-32bd-49ab-b58b-d14255763990 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:51.064902 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3c62683c-32bd-49ab-b58b-d14255763990 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:51.065336 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3c62683c-32bd-49ab-b58b-d14255763990 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:51.066105 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3c62683c-32bd-49ab-b58b-d14255763990 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:51.066574 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3c62683c-32bd-49ab-b58b-d14255763990 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:51.066978 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3c62683c-32bd-49ab-b58b-d14255763990 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:51.305471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-3c62683c-32bd-49ab-b58b-d14255763990 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:51.313362 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-3c62683c-32bd-49ab-b58b-d14255763990 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1266 microversion: 2.1 time: 0.260456 Oct 24 09:00:51.313929 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 464/922] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:51 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1266 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:52.337821 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-15cc22fd-0819-4470-922b-97bc86266d72 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:52.338746 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-15cc22fd-0819-4470-922b-97bc86266d72 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:52.346357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-15cc22fd-0819-4470-922b-97bc86266d72 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:52.346878 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-15cc22fd-0819-4470-922b-97bc86266d72 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:52.347388 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-15cc22fd-0819-4470-922b-97bc86266d72 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:52.348198 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-15cc22fd-0819-4470-922b-97bc86266d72 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:52.348748 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-15cc22fd-0819-4470-922b-97bc86266d72 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:52.349219 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-15cc22fd-0819-4470-922b-97bc86266d72 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:52.490314 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-15cc22fd-0819-4470-922b-97bc86266d72 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:52.498679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-15cc22fd-0819-4470-922b-97bc86266d72 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1266 microversion: 2.1 time: 0.165429 Oct 24 09:00:52.499272 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 459/923] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:52 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1266 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:52.767162 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7a5a2c61-ceca-4313-ac7d-9c9c8b7755ba tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:52.767747 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7a5a2c61-ceca-4313-ac7d-9c9c8b7755ba tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:52.774056 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7a5a2c61-ceca-4313-ac7d-9c9c8b7755ba tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:52.774794 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7a5a2c61-ceca-4313-ac7d-9c9c8b7755ba tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:52.775227 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7a5a2c61-ceca-4313-ac7d-9c9c8b7755ba tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:52.775923 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7a5a2c61-ceca-4313-ac7d-9c9c8b7755ba tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:52.776409 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7a5a2c61-ceca-4313-ac7d-9c9c8b7755ba tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:52.776841 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7a5a2c61-ceca-4313-ac7d-9c9c8b7755ba tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:52.864030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-7a5a2c61-ceca-4313-ac7d-9c9c8b7755ba tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lazy-loading 'services' on Instance uuid 735ff4f8-a704-4d00-b8f0-3d13e88bb28c {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 09:00:52.935380 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7a5a2c61-ceca-4313-ac7d-9c9c8b7755ba tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "DELETE /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c/os-volume_attachments/4f402f7d-ed05-425d-9e70-ad22d5f86615" status: 202 len: 0 microversion: 2.1 time: 0.175538 Oct 24 09:00:52.936215 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 465/924] 10.222.0.39 () {58 vars in 1416 bytes} [Mon Oct 24 09:00:52 2022] DELETE /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c/os-volume_attachments/4f402f7d-ed05-425d-9e70-ad22d5f86615 => generated 0 bytes in 177 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 24 09:00:53.522700 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e2c881c7-b591-4ae1-bc56-0bada979457a tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:53.523343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e2c881c7-b591-4ae1-bc56-0bada979457a tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:53.530823 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e2c881c7-b591-4ae1-bc56-0bada979457a tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:53.531312 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e2c881c7-b591-4ae1-bc56-0bada979457a tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:53.531750 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e2c881c7-b591-4ae1-bc56-0bada979457a tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:53.532487 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e2c881c7-b591-4ae1-bc56-0bada979457a tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:53.532945 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e2c881c7-b591-4ae1-bc56-0bada979457a tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:53.533372 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e2c881c7-b591-4ae1-bc56-0bada979457a tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:53.723085 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e2c881c7-b591-4ae1-bc56-0bada979457a tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:53.731027 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e2c881c7-b591-4ae1-bc56-0bada979457a tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1266 microversion: 2.1 time: 0.212044 Oct 24 09:00:53.731591 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 460/925] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:53 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1266 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:53.998207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-33672b70-f9c8-4b6f-a982-583675b621c2 req-ce197724-90fc-451b-bdad-ef99fba36c79 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f2c4ad60-dd40-42b5-8728-8a414e8d7644", "name": "network-vif-plugged", "status": "completed", "tag": "ea79d810-db9b-4fba-9d68-f8b0560784ee"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:00:54.010549 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-33672b70-f9c8-4b6f-a982-583675b621c2 req-ce197724-90fc-451b-bdad-ef99fba36c79 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:54.011509 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-33672b70-f9c8-4b6f-a982-583675b621c2 req-ce197724-90fc-451b-bdad-ef99fba36c79 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:54.012765 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-33672b70-f9c8-4b6f-a982-583675b621c2 req-ce197724-90fc-451b-bdad-ef99fba36c79 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:54.024113 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-33672b70-f9c8-4b6f-a982-583675b621c2 req-ce197724-90fc-451b-bdad-ef99fba36c79 service nova] Creating event network-vif-plugged:ea79d810-db9b-4fba-9d68-f8b0560784ee for instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 on hv-cinder-maste Oct 24 09:00:54.029858 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-33672b70-f9c8-4b6f-a982-583675b621c2 req-ce197724-90fc-451b-bdad-ef99fba36c79 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037379 Oct 24 09:00:54.031227 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 466/926] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:00:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:00:54.754416 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5a08a11f-1476-4f66-af51-1c85967150ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:54.755073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-5a08a11f-1476-4f66-af51-1c85967150ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:54.761887 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5a08a11f-1476-4f66-af51-1c85967150ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:54.762376 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5a08a11f-1476-4f66-af51-1c85967150ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:54.762806 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5a08a11f-1476-4f66-af51-1c85967150ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:54.763528 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5a08a11f-1476-4f66-af51-1c85967150ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:54.763974 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5a08a11f-1476-4f66-af51-1c85967150ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:54.764430 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5a08a11f-1476-4f66-af51-1c85967150ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:54.894109 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5a08a11f-1476-4f66-af51-1c85967150ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:54.902682 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5a08a11f-1476-4f66-af51-1c85967150ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1266 microversion: 2.1 time: 0.151569 Oct 24 09:00:54.903556 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 461/927] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:54 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1266 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:55.138225 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-91974069-5754-43a8-9492-01dddd66d771 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:55.138808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-91974069-5754-43a8-9492-01dddd66d771 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:55.145427 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91974069-5754-43a8-9492-01dddd66d771 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:55.146043 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91974069-5754-43a8-9492-01dddd66d771 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:55.146510 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91974069-5754-43a8-9492-01dddd66d771 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:55.147261 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91974069-5754-43a8-9492-01dddd66d771 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:55.147747 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91974069-5754-43a8-9492-01dddd66d771 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:55.148272 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-91974069-5754-43a8-9492-01dddd66d771 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:55.173807 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-91974069-5754-43a8-9492-01dddd66d771 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Going to try to terminate instance {{(pid=130395) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 09:00:55.248906 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-91974069-5754-43a8-9492-01dddd66d771 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "DELETE /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 204 len: 0 microversion: 2.1 time: 0.114722 Oct 24 09:00:55.250642 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 467/928] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 09:00:55 2022] DELETE /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 09:00:55.258892 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a195a6d9-5d75-414b-830d-d045186b6e34 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:55.259536 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-a195a6d9-5d75-414b-830d-d045186b6e34 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:55.265861 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a195a6d9-5d75-414b-830d-d045186b6e34 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:55.266367 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a195a6d9-5d75-414b-830d-d045186b6e34 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:55.266837 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a195a6d9-5d75-414b-830d-d045186b6e34 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:55.267555 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a195a6d9-5d75-414b-830d-d045186b6e34 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:55.268044 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a195a6d9-5d75-414b-830d-d045186b6e34 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:55.268500 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a195a6d9-5d75-414b-830d-d045186b6e34 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:55.401702 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-a195a6d9-5d75-414b-830d-d045186b6e34 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:55.410204 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-a195a6d9-5d75-414b-830d-d045186b6e34 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 200 len: 1514 microversion: 2.1 time: 0.155127 Oct 24 09:00:55.410778 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 462/929] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:55 2022] GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 1514 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:55.927317 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-38b1a94d-4be1-4f58-a5ad-d4ab63f681f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:55.928056 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-38b1a94d-4be1-4f58-a5ad-d4ab63f681f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:55.934033 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-38b1a94d-4be1-4f58-a5ad-d4ab63f681f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:55.934544 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-38b1a94d-4be1-4f58-a5ad-d4ab63f681f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:55.934980 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-38b1a94d-4be1-4f58-a5ad-d4ab63f681f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:55.935670 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-38b1a94d-4be1-4f58-a5ad-d4ab63f681f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:55.936123 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-38b1a94d-4be1-4f58-a5ad-d4ab63f681f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:55.936567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-38b1a94d-4be1-4f58-a5ad-d4ab63f681f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:55.940601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:00:56.072197 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-38b1a94d-4be1-4f58-a5ad-d4ab63f681f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:56.080649 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-38b1a94d-4be1-4f58-a5ad-d4ab63f681f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1266 microversion: 2.1 time: 0.157547 Oct 24 09:00:56.081218 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 468/930] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:55 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1266 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:56.434963 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-228abb1f-ebe7-4185-a352-fab6a66e8780 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:56.435563 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-228abb1f-ebe7-4185-a352-fab6a66e8780 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:56.441957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-228abb1f-ebe7-4185-a352-fab6a66e8780 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:56.442434 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-228abb1f-ebe7-4185-a352-fab6a66e8780 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:56.442862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-228abb1f-ebe7-4185-a352-fab6a66e8780 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:56.443550 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-228abb1f-ebe7-4185-a352-fab6a66e8780 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:56.444001 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-228abb1f-ebe7-4185-a352-fab6a66e8780 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:56.444458 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-228abb1f-ebe7-4185-a352-fab6a66e8780 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:56.446911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:00:56.576906 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-228abb1f-ebe7-4185-a352-fab6a66e8780 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:56.584565 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-228abb1f-ebe7-4185-a352-fab6a66e8780 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 200 len: 1514 microversion: 2.1 time: 0.153938 Oct 24 09:00:56.585174 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 463/931] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:56 2022] GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 1514 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:57.103539 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-fedb6e87-0cb7-43e0-bf3d-6c6165cc51bb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:57.104447 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-fedb6e87-0cb7-43e0-bf3d-6c6165cc51bb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:57.113215 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fedb6e87-0cb7-43e0-bf3d-6c6165cc51bb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:57.113917 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fedb6e87-0cb7-43e0-bf3d-6c6165cc51bb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:57.114547 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fedb6e87-0cb7-43e0-bf3d-6c6165cc51bb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:57.115391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fedb6e87-0cb7-43e0-bf3d-6c6165cc51bb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:57.115829 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fedb6e87-0cb7-43e0-bf3d-6c6165cc51bb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:57.116289 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fedb6e87-0cb7-43e0-bf3d-6c6165cc51bb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:57.256462 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-fedb6e87-0cb7-43e0-bf3d-6c6165cc51bb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:57.264648 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-fedb6e87-0cb7-43e0-bf3d-6c6165cc51bb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1266 microversion: 2.1 time: 0.165163 Oct 24 09:00:57.266200 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 469/932] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:57 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1266 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:57.609402 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ba09e703-e95e-46f7-bf96-5110d0a8c9d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:57.609981 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ba09e703-e95e-46f7-bf96-5110d0a8c9d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:57.612251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=34 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:00:57.617819 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ba09e703-e95e-46f7-bf96-5110d0a8c9d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:57.618312 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ba09e703-e95e-46f7-bf96-5110d0a8c9d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:57.618759 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ba09e703-e95e-46f7-bf96-5110d0a8c9d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:57.619460 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ba09e703-e95e-46f7-bf96-5110d0a8c9d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:57.619918 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ba09e703-e95e-46f7-bf96-5110d0a8c9d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:57.620378 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ba09e703-e95e-46f7-bf96-5110d0a8c9d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:57.753151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ba09e703-e95e-46f7-bf96-5110d0a8c9d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:57.760862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ba09e703-e95e-46f7-bf96-5110d0a8c9d8 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 200 len: 1514 microversion: 2.1 time: 0.156207 Oct 24 09:00:57.762738 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 464/933] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:57 2022] GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 1514 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:58.288012 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c8c1736c-9046-4c73-86f2-5583a5d81b53 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:58.289233 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c8c1736c-9046-4c73-86f2-5583a5d81b53 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:58.296548 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c8c1736c-9046-4c73-86f2-5583a5d81b53 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:58.297041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c8c1736c-9046-4c73-86f2-5583a5d81b53 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:58.297481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c8c1736c-9046-4c73-86f2-5583a5d81b53 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:58.298194 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c8c1736c-9046-4c73-86f2-5583a5d81b53 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:58.298647 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c8c1736c-9046-4c73-86f2-5583a5d81b53 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:58.299075 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c8c1736c-9046-4c73-86f2-5583a5d81b53 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:58.439659 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c8c1736c-9046-4c73-86f2-5583a5d81b53 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:58.447604 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c8c1736c-9046-4c73-86f2-5583a5d81b53 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1453 microversion: 2.1 time: 0.163476 Oct 24 09:00:58.448195 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 470/934] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:58 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1453 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:58.471539 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b515f99c-11a9-4dbd-be06-8342f4e67ba4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:58.472272 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b515f99c-11a9-4dbd-be06-8342f4e67ba4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:58.481069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b515f99c-11a9-4dbd-be06-8342f4e67ba4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:58.481069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b515f99c-11a9-4dbd-be06-8342f4e67ba4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:58.481069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b515f99c-11a9-4dbd-be06-8342f4e67ba4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:58.481069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b515f99c-11a9-4dbd-be06-8342f4e67ba4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:58.481733 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b515f99c-11a9-4dbd-be06-8342f4e67ba4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:58.481914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b515f99c-11a9-4dbd-be06-8342f4e67ba4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:58.519805 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-1b736d5b-2fcc-4adb-a4cb-ea47f243a659 req-9d2a3f36-8033-4e23-b7ac-48e38b070b45 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "735ff4f8-a704-4d00-b8f0-3d13e88bb28c", "name": "network-vif-deleted", "tag": "8b267f47-1042-4faa-870e-5e525d61f3d4"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:00:58.530593 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-1b736d5b-2fcc-4adb-a4cb-ea47f243a659 req-9d2a3f36-8033-4e23-b7ac-48e38b070b45 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:58.531514 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-1b736d5b-2fcc-4adb-a4cb-ea47f243a659 req-9d2a3f36-8033-4e23-b7ac-48e38b070b45 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:58.531993 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-1b736d5b-2fcc-4adb-a4cb-ea47f243a659 req-9d2a3f36-8033-4e23-b7ac-48e38b070b45 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:58.547341 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-1b736d5b-2fcc-4adb-a4cb-ea47f243a659 req-9d2a3f36-8033-4e23-b7ac-48e38b070b45 service nova] Creating event network-vif-deleted:8b267f47-1042-4faa-870e-5e525d61f3d4 for instance 735ff4f8-a704-4d00-b8f0-3d13e88bb28c on hv-cinder-maste Oct 24 09:00:58.552239 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-1b736d5b-2fcc-4adb-a4cb-ea47f243a659 req-9d2a3f36-8033-4e23-b7ac-48e38b070b45 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035849 Oct 24 09:00:58.552994 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 471/935] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:00:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:00:58.635766 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b515f99c-11a9-4dbd-be06-8342f4e67ba4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:58.644146 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b515f99c-11a9-4dbd-be06-8342f4e67ba4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1453 microversion: 2.1 time: 0.176436 Oct 24 09:00:58.644737 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 465/936] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:58 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1453 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:58.786128 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-3d839cc5-b0b4-43e3-ada2-b6314466b242 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:58.786858 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-3d839cc5-b0b4-43e3-ada2-b6314466b242 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:58.789038 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=54,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:00:58.794904 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d839cc5-b0b4-43e3-ada2-b6314466b242 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:58.795394 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d839cc5-b0b4-43e3-ada2-b6314466b242 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:58.796016 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d839cc5-b0b4-43e3-ada2-b6314466b242 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:58.796746 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d839cc5-b0b4-43e3-ada2-b6314466b242 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:58.797360 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d839cc5-b0b4-43e3-ada2-b6314466b242 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:58.797791 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d839cc5-b0b4-43e3-ada2-b6314466b242 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:58.832496 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d839cc5-b0b4-43e3-ada2-b6314466b242 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:58.832975 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d839cc5-b0b4-43e3-ada2-b6314466b242 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:58.833464 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d839cc5-b0b4-43e3-ada2-b6314466b242 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:58.834316 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-3d839cc5-b0b4-43e3-ada2-b6314466b242 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lazy-loading 'fault' on Instance uuid 735ff4f8-a704-4d00-b8f0-3d13e88bb28c {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 09:00:58.877746 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-3d839cc5-b0b4-43e3-ada2-b6314466b242 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c94f6e2fa2b94630a745af7c54ec680a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01318d53c6ed499e94a25e12f826719f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:00:58.885791 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-3d839cc5-b0b4-43e3-ada2-b6314466b242 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 200 len: 1309 microversion: 2.1 time: 0.104425 Oct 24 09:00:58.887781 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 472/937] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:58 2022] GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 1309 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:00:59.911116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-196629b7-8b27-43a2-a412-41cb7104bbf2 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:00:59.911761 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-196629b7-8b27-43a2-a412-41cb7104bbf2 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] [instance: 735ff4f8-a704-4d00-b8f0-3d13e88bb28c] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:00:59.919267 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-196629b7-8b27-43a2-a412-41cb7104bbf2 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:59.919758 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-196629b7-8b27-43a2-a412-41cb7104bbf2 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:59.920224 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-196629b7-8b27-43a2-a412-41cb7104bbf2 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:59.920938 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-196629b7-8b27-43a2-a412-41cb7104bbf2 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:00:59.921390 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-196629b7-8b27-43a2-a412-41cb7104bbf2 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:00:59.921816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-196629b7-8b27-43a2-a412-41cb7104bbf2 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:00:59.936111 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-196629b7-8b27-43a2-a412-41cb7104bbf2 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] HTTP exception thrown: Instance 735ff4f8-a704-4d00-b8f0-3d13e88bb28c could not be found. Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-196629b7-8b27-43a2-a412-41cb7104bbf2 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Request method failure captured: Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c HTTP/1.1 Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlO1a4gxS_RX3u8K2x4yhOW9vrETflXSD23tAtVBS7hkMwBWoZeh-Y2ZlFaHsFODJyK_i6NcFeyOGykc4y3wOcVrsevFrHzoHRExS-f_cS-PyU4duotodp4i2nhk5ZvdtlDjgwPWraqFzelYlfiLV20uqQwKz_c-_qjQoZ7KagjfL3U Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 01318d53c6ed499e94a25e12f826719f Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-VolumesSnapshotTestJSON-68320019 Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 09:00:59.937805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 09:00:59.939517 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/01318d53c6ed499e94a25e12f826719f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_01318d53c6ed499e94a25e12f826719f", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/01318d53c6ed499e94a25e12f826719f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/01318d53c6ed499e94a25e12f826719f"}]}] Oct 24 09:00:59.939517 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-VolumesSnapshotTestJSON-68320019 Oct 24 09:00:59.939517 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 01318d53c6ed499e94a25e12f826719f Oct 24 09:00:59.939517 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-68320019 Oct 24 09:00:59.939517 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-VolumesSnapshotTestJSON-68320019-project Oct 24 09:00:59.939517 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 09:00:59.939517 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 09:00:59.939517 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: c94f6e2fa2b94630a745af7c54ec680a Oct 24 09:00:59.939517 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-VolumesSnapshotTestJSON-68320019-project Oct 24 09:00:59.939517 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 09:00:59.939517 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance 735ff4f8-a704-4d00-b8f0-3d13e88bb28c could not be found. Oct 24 09:00:59.939517 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': '735ff4f8-a704-4d00-b8f0-3d13e88bb28c'} Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 735ff4f8-a704-4d00-b8f0-3d13e88bb28c could not be found. Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:00:59.943541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 735ff4f8-a704-4d00-b8f0-3d13e88bb28c could not be found. Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 735ff4f8-a704-4d00-b8f0-3d13e88bb28c could not be found. Oct 24 09:00:59.945205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:00:59.946643 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-196629b7-8b27-43a2-a412-41cb7104bbf2 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] Returning 404 to user: Instance 735ff4f8-a704-4d00-b8f0-3d13e88bb28c could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:00:59.946643 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-196629b7-8b27-43a2-a412-41cb7104bbf2 tempest-VolumesSnapshotTestJSON-68320019 tempest-VolumesSnapshotTestJSON-68320019-project] 10.222.0.39 "GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c" status: 404 len: 111 microversion: 2.1 time: 0.032139 Oct 24 09:00:59.946643 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 466/938] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:00:59 2022] GET /compute/v2.1/servers/735ff4f8-a704-4d00-b8f0-3d13e88bb28c => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:01:00.567309 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-660830b3-5181-4331-9beb-0c88ce566212 req-beaa6cf7-b720-4063-9422-b236e18f73d9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f2c4ad60-dd40-42b5-8728-8a414e8d7644", "tag": "ea79d810-db9b-4fba-9d68-f8b0560784ee"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:01:00.578398 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-660830b3-5181-4331-9beb-0c88ce566212 req-beaa6cf7-b720-4063-9422-b236e18f73d9 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:00.578879 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-660830b3-5181-4331-9beb-0c88ce566212 req-beaa6cf7-b720-4063-9422-b236e18f73d9 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:00.579316 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-660830b3-5181-4331-9beb-0c88ce566212 req-beaa6cf7-b720-4063-9422-b236e18f73d9 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:00.592082 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-660830b3-5181-4331-9beb-0c88ce566212 req-beaa6cf7-b720-4063-9422-b236e18f73d9 service nova] Creating event network-changed:ea79d810-db9b-4fba-9d68-f8b0560784ee for instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 on hv-cinder-maste Oct 24 09:01:00.597739 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-660830b3-5181-4331-9beb-0c88ce566212 req-beaa6cf7-b720-4063-9422-b236e18f73d9 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033908 Oct 24 09:01:00.598343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 473/939] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:01:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:01:03.878635 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-def06d37-7d5e-4f72-896f-5a51353ea1c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:03.879299 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-def06d37-7d5e-4f72-896f-5a51353ea1c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:03.885448 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-def06d37-7d5e-4f72-896f-5a51353ea1c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:03.886563 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-def06d37-7d5e-4f72-896f-5a51353ea1c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:03.887039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-def06d37-7d5e-4f72-896f-5a51353ea1c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:03.887835 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-def06d37-7d5e-4f72-896f-5a51353ea1c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:03.888386 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-def06d37-7d5e-4f72-896f-5a51353ea1c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:03.888842 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-def06d37-7d5e-4f72-896f-5a51353ea1c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:03.912301 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-def06d37-7d5e-4f72-896f-5a51353ea1c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/ips" status: 200 len: 145 microversion: 2.1 time: 0.037947 Oct 24 09:01:03.912868 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 467/940] 10.222.0.39 () {58 vars in 1248 bytes} [Mon Oct 24 09:01:03 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/ips => generated 145 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:01:06.317556 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-26c7fa92-6b6e-4dbb-8ca8-bf6e030ce0c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "44d7425f-0e58-4ca3-8910-fe2ace862106"}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:01:06.320040 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-26c7fa92-6b6e-4dbb-8ca8-bf6e030ce0c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:06.327080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26c7fa92-6b6e-4dbb-8ca8-bf6e030ce0c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:06.327560 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26c7fa92-6b6e-4dbb-8ca8-bf6e030ce0c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:06.328000 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26c7fa92-6b6e-4dbb-8ca8-bf6e030ce0c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:06.328729 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26c7fa92-6b6e-4dbb-8ca8-bf6e030ce0c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:06.329174 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26c7fa92-6b6e-4dbb-8ca8-bf6e030ce0c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:06.329605 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26c7fa92-6b6e-4dbb-8ca8-bf6e030ce0c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:06.892411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-26c7fa92-6b6e-4dbb-8ca8-bf6e030ce0c7 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "POST /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.578945 Oct 24 09:01:06.893019 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 474/941] 10.222.0.39 () {60 vars in 1323 bytes} [Mon Oct 24 09:01:06 2022] POST /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments => generated 194 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:01:17.637521 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-977db0b2-cb37-4a0c-bdac-d3513a723214 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:17.639809 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-977db0b2-cb37-4a0c-bdac-d3513a723214 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:17.642321 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=7 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:01:17.643005 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=23 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:01:17.649060 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-977db0b2-cb37-4a0c-bdac-d3513a723214 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:17.649559 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-977db0b2-cb37-4a0c-bdac-d3513a723214 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:17.649998 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-977db0b2-cb37-4a0c-bdac-d3513a723214 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:17.650702 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-977db0b2-cb37-4a0c-bdac-d3513a723214 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:17.651234 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-977db0b2-cb37-4a0c-bdac-d3513a723214 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:17.651786 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-977db0b2-cb37-4a0c-bdac-d3513a723214 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:17.682291 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-977db0b2-cb37-4a0c-bdac-d3513a723214 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049670 Oct 24 09:01:17.683441 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 468/942] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 09:01:17 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:01:17.691902 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=7 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:01:17.692719 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=40,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:01:17.697066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-346ea907-1855-461f-9ca1-62bcb42511c0 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:17.697674 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-346ea907-1855-461f-9ca1-62bcb42511c0 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:17.705691 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-346ea907-1855-461f-9ca1-62bcb42511c0 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:17.706172 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-346ea907-1855-461f-9ca1-62bcb42511c0 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:17.706611 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-346ea907-1855-461f-9ca1-62bcb42511c0 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:17.707311 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-346ea907-1855-461f-9ca1-62bcb42511c0 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:17.707763 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-346ea907-1855-461f-9ca1-62bcb42511c0 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:17.708208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-346ea907-1855-461f-9ca1-62bcb42511c0 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:17.737799 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-346ea907-1855-461f-9ca1-62bcb42511c0 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments/44d7425f-0e58-4ca3-8910-fe2ace862106" status: 200 len: 194 microversion: 2.1 time: 0.047392 Oct 24 09:01:17.738651 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 475/943] 10.222.0.39 () {58 vars in 1413 bytes} [Mon Oct 24 09:01:17 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments/44d7425f-0e58-4ca3-8910-fe2ace862106 => generated 194 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:01:19.080048 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-443270a8-3a36-4af6-a09c-2a358630f81e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f63808d9-4027-4aac-847c-dd55f54bf228"}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:01:19.082648 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-443270a8-3a36-4af6-a09c-2a358630f81e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:19.090071 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-443270a8-3a36-4af6-a09c-2a358630f81e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:19.090562 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-443270a8-3a36-4af6-a09c-2a358630f81e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:19.091007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-443270a8-3a36-4af6-a09c-2a358630f81e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:19.091747 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-443270a8-3a36-4af6-a09c-2a358630f81e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:19.092233 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-443270a8-3a36-4af6-a09c-2a358630f81e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:19.092668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-443270a8-3a36-4af6-a09c-2a358630f81e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:19.371537 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-443270a8-3a36-4af6-a09c-2a358630f81e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "POST /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.293755 Oct 24 09:01:19.373906 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 469/944] 10.222.0.39 () {60 vars in 1323 bytes} [Mon Oct 24 09:01:19 2022] POST /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments => generated 194 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:01:26.945290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-bf3493ca-eecd-4aee-9c68-8bf677c41cf5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:26.949043 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-bf3493ca-eecd-4aee-9c68-8bf677c41cf5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:26.959823 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bf3493ca-eecd-4aee-9c68-8bf677c41cf5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:26.959823 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bf3493ca-eecd-4aee-9c68-8bf677c41cf5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:26.959823 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bf3493ca-eecd-4aee-9c68-8bf677c41cf5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:26.959823 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bf3493ca-eecd-4aee-9c68-8bf677c41cf5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:26.960380 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bf3493ca-eecd-4aee-9c68-8bf677c41cf5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:26.960380 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-bf3493ca-eecd-4aee-9c68-8bf677c41cf5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:26.990769 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-bf3493ca-eecd-4aee-9c68-8bf677c41cf5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.049371 Oct 24 09:01:26.991674 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 476/945] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 09:01:26 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments => generated 371 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:01:27.136123 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-82b855c3-01f8-4977-be09-d96c010ae6c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:27.136989 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-82b855c3-01f8-4977-be09-d96c010ae6c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:27.143277 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-82b855c3-01f8-4977-be09-d96c010ae6c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:27.143767 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-82b855c3-01f8-4977-be09-d96c010ae6c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:27.144236 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-82b855c3-01f8-4977-be09-d96c010ae6c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:27.144941 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-82b855c3-01f8-4977-be09-d96c010ae6c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:27.145400 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-82b855c3-01f8-4977-be09-d96c010ae6c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:27.145830 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-82b855c3-01f8-4977-be09-d96c010ae6c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:27.175955 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-82b855c3-01f8-4977-be09-d96c010ae6c5 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments/44d7425f-0e58-4ca3-8910-fe2ace862106" status: 200 len: 194 microversion: 2.1 time: 0.045404 Oct 24 09:01:27.176631 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 470/946] 10.222.0.39 () {58 vars in 1413 bytes} [Mon Oct 24 09:01:27 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments/44d7425f-0e58-4ca3-8910-fe2ace862106 => generated 194 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:01:27.187744 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-aa7f1ace-a77a-455f-a365-a2751269fe75 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:27.188294 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-aa7f1ace-a77a-455f-a365-a2751269fe75 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:27.194382 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa7f1ace-a77a-455f-a365-a2751269fe75 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:27.194879 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa7f1ace-a77a-455f-a365-a2751269fe75 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:27.195322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa7f1ace-a77a-455f-a365-a2751269fe75 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:27.196024 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa7f1ace-a77a-455f-a365-a2751269fe75 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:27.196508 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa7f1ace-a77a-455f-a365-a2751269fe75 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:27.196937 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aa7f1ace-a77a-455f-a365-a2751269fe75 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:27.281078 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-aa7f1ace-a77a-455f-a365-a2751269fe75 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lazy-loading 'services' on Instance uuid 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 09:01:27.361834 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-aa7f1ace-a77a-455f-a365-a2751269fe75 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "DELETE /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments/44d7425f-0e58-4ca3-8910-fe2ace862106" status: 202 len: 0 microversion: 2.1 time: 0.177561 Oct 24 09:01:27.363218 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 477/947] 10.222.0.39 () {58 vars in 1416 bytes} [Mon Oct 24 09:01:27 2022] DELETE /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments/44d7425f-0e58-4ca3-8910-fe2ace862106 => generated 0 bytes in 181 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 24 09:01:29.603856 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-62968919-903f-4a17-931a-0f27abce65b2 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:29.605297 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-62968919-903f-4a17-931a-0f27abce65b2 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:29.612273 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62968919-903f-4a17-931a-0f27abce65b2 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:29.612842 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62968919-903f-4a17-931a-0f27abce65b2 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:29.613384 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62968919-903f-4a17-931a-0f27abce65b2 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:29.614445 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62968919-903f-4a17-931a-0f27abce65b2 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:29.614969 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62968919-903f-4a17-931a-0f27abce65b2 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:29.615468 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62968919-903f-4a17-931a-0f27abce65b2 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:29.650804 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-62968919-903f-4a17-931a-0f27abce65b2 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments/f63808d9-4027-4aac-847c-dd55f54bf228" status: 200 len: 194 microversion: 2.1 time: 0.052157 Oct 24 09:01:29.651586 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 471/948] 10.222.0.39 () {58 vars in 1413 bytes} [Mon Oct 24 09:01:29 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments/f63808d9-4027-4aac-847c-dd55f54bf228 => generated 194 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:01:29.665443 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ba2d9f39-906f-4ed1-b058-7787650954d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:29.666155 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ba2d9f39-906f-4ed1-b058-7787650954d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:29.674751 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ba2d9f39-906f-4ed1-b058-7787650954d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:29.675237 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ba2d9f39-906f-4ed1-b058-7787650954d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:29.675664 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ba2d9f39-906f-4ed1-b058-7787650954d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:29.676582 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ba2d9f39-906f-4ed1-b058-7787650954d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:29.677033 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ba2d9f39-906f-4ed1-b058-7787650954d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:29.677454 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ba2d9f39-906f-4ed1-b058-7787650954d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:29.781498 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-ba2d9f39-906f-4ed1-b058-7787650954d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lazy-loading 'services' on Instance uuid 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 09:01:29.881482 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ba2d9f39-906f-4ed1-b058-7787650954d1 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "DELETE /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments/f63808d9-4027-4aac-847c-dd55f54bf228" status: 202 len: 0 microversion: 2.1 time: 0.222394 Oct 24 09:01:29.882042 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 478/949] 10.222.0.39 () {58 vars in 1416 bytes} [Mon Oct 24 09:01:29 2022] DELETE /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments/f63808d9-4027-4aac-847c-dd55f54bf228 => generated 0 bytes in 224 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 09:01:30.040944 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ac56d0e6-ba80-4c9a-a322-8974d30581d9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:30.040944 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ac56d0e6-ba80-4c9a-a322-8974d30581d9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:30.046485 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ac56d0e6-ba80-4c9a-a322-8974d30581d9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:30.047078 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ac56d0e6-ba80-4c9a-a322-8974d30581d9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:30.047614 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ac56d0e6-ba80-4c9a-a322-8974d30581d9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:30.048426 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ac56d0e6-ba80-4c9a-a322-8974d30581d9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:30.053340 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ac56d0e6-ba80-4c9a-a322-8974d30581d9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:30.053801 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ac56d0e6-ba80-4c9a-a322-8974d30581d9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:30.088823 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ac56d0e6-ba80-4c9a-a322-8974d30581d9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853/ips" status: 200 len: 150 microversion: 2.1 time: 0.394061 Oct 24 09:01:30.089675 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 472/950] 10.222.0.39 () {58 vars in 1248 bytes} [Mon Oct 24 09:01:29 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853/ips => generated 150 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 09:01:31.528053 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a97f054b-b3ff-44d1-8519-e2e5544473d4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3f302bb0-a01d-41ae-b11c-26bae986ef75"}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:01:31.530838 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a97f054b-b3ff-44d1-8519-e2e5544473d4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:31.543073 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a97f054b-b3ff-44d1-8519-e2e5544473d4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:31.548811 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a97f054b-b3ff-44d1-8519-e2e5544473d4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:31.549305 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a97f054b-b3ff-44d1-8519-e2e5544473d4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:31.550113 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a97f054b-b3ff-44d1-8519-e2e5544473d4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:31.550580 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a97f054b-b3ff-44d1-8519-e2e5544473d4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:31.551017 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a97f054b-b3ff-44d1-8519-e2e5544473d4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:31.906355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a97f054b-b3ff-44d1-8519-e2e5544473d4 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "POST /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.384404 Oct 24 09:01:31.915636 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 479/951] 10.222.0.39 () {60 vars in 1323 bytes} [Mon Oct 24 09:01:31 2022] POST /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853/os-volume_attachments => generated 194 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:01:32.261426 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f7ab68b3-7dee-4a77-a0c1-076206bfcfa3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:32.264469 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f7ab68b3-7dee-4a77-a0c1-076206bfcfa3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:32.272226 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7ab68b3-7dee-4a77-a0c1-076206bfcfa3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:32.272650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7ab68b3-7dee-4a77-a0c1-076206bfcfa3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:32.273095 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7ab68b3-7dee-4a77-a0c1-076206bfcfa3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:32.273810 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7ab68b3-7dee-4a77-a0c1-076206bfcfa3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:32.274812 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7ab68b3-7dee-4a77-a0c1-076206bfcfa3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:32.274812 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f7ab68b3-7dee-4a77-a0c1-076206bfcfa3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:32.308088 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f7ab68b3-7dee-4a77-a0c1-076206bfcfa3 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.050141 Oct 24 09:01:32.308699 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 473/952] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 09:01:32 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments => generated 25 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 24 09:01:32.438772 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-65ae4901-8e87-4bea-9d3b-378932fc054e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:32.441151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-65ae4901-8e87-4bea-9d3b-378932fc054e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:32.450899 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-65ae4901-8e87-4bea-9d3b-378932fc054e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:32.451800 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-65ae4901-8e87-4bea-9d3b-378932fc054e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:32.452753 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-65ae4901-8e87-4bea-9d3b-378932fc054e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:32.453681 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-65ae4901-8e87-4bea-9d3b-378932fc054e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:32.454591 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-65ae4901-8e87-4bea-9d3b-378932fc054e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:32.455079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-65ae4901-8e87-4bea-9d3b-378932fc054e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:32.500876 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-65ae4901-8e87-4bea-9d3b-378932fc054e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.066510 Oct 24 09:01:32.501459 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 480/953] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 09:01:32 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3/os-volume_attachments => generated 25 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 24 09:01:32.587962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-d044970c-1fdf-48d0-9fda-65603aef3b0d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:32.589740 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-d044970c-1fdf-48d0-9fda-65603aef3b0d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:32.597598 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d044970c-1fdf-48d0-9fda-65603aef3b0d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:32.598211 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d044970c-1fdf-48d0-9fda-65603aef3b0d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:32.598693 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d044970c-1fdf-48d0-9fda-65603aef3b0d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:32.599673 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d044970c-1fdf-48d0-9fda-65603aef3b0d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:32.600390 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d044970c-1fdf-48d0-9fda-65603aef3b0d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:32.600967 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d044970c-1fdf-48d0-9fda-65603aef3b0d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:32.640436 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-d044970c-1fdf-48d0-9fda-65603aef3b0d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Going to try to terminate instance {{(pid=130396) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 09:01:32.741398 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-d044970c-1fdf-48d0-9fda-65603aef3b0d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "DELETE /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 204 len: 0 microversion: 2.1 time: 0.157193 Oct 24 09:01:32.744011 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 474/954] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 09:01:32 2022] DELETE /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 09:01:32.752159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-4862791d-b4e8-4d04-9d7a-dbaccfcc8c3d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:32.753851 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-4862791d-b4e8-4d04-9d7a-dbaccfcc8c3d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:32.763047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4862791d-b4e8-4d04-9d7a-dbaccfcc8c3d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:32.763909 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4862791d-b4e8-4d04-9d7a-dbaccfcc8c3d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:32.764446 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4862791d-b4e8-4d04-9d7a-dbaccfcc8c3d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:32.765404 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4862791d-b4e8-4d04-9d7a-dbaccfcc8c3d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:32.766275 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4862791d-b4e8-4d04-9d7a-dbaccfcc8c3d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:32.767080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4862791d-b4e8-4d04-9d7a-dbaccfcc8c3d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:32.942158 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-4862791d-b4e8-4d04-9d7a-dbaccfcc8c3d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:32.950154 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-4862791d-b4e8-4d04-9d7a-dbaccfcc8c3d tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1671 microversion: 2.1 time: 0.201665 Oct 24 09:01:32.950961 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 481/955] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:32 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1671 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:01:33.978754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-834f2022-ddb1-4e39-a507-3cd0bda22eb6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:33.981578 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-834f2022-ddb1-4e39-a507-3cd0bda22eb6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:33.994153 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-834f2022-ddb1-4e39-a507-3cd0bda22eb6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:33.994153 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-834f2022-ddb1-4e39-a507-3cd0bda22eb6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:33.994525 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-834f2022-ddb1-4e39-a507-3cd0bda22eb6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:33.996007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-834f2022-ddb1-4e39-a507-3cd0bda22eb6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:33.998231 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-834f2022-ddb1-4e39-a507-3cd0bda22eb6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:33.998679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-834f2022-ddb1-4e39-a507-3cd0bda22eb6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:34.200702 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-834f2022-ddb1-4e39-a507-3cd0bda22eb6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:34.208399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-834f2022-ddb1-4e39-a507-3cd0bda22eb6 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1671 microversion: 2.1 time: 0.237674 Oct 24 09:01:34.224570 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 475/956] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:33 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1671 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:01:35.233528 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-40bb9caa-3c08-4b24-86d8-8c5d6e279f4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:35.234886 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-40bb9caa-3c08-4b24-86d8-8c5d6e279f4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:35.246967 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-40bb9caa-3c08-4b24-86d8-8c5d6e279f4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:35.249382 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-40bb9caa-3c08-4b24-86d8-8c5d6e279f4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:35.250063 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-40bb9caa-3c08-4b24-86d8-8c5d6e279f4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:35.251913 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-40bb9caa-3c08-4b24-86d8-8c5d6e279f4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:35.253106 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-40bb9caa-3c08-4b24-86d8-8c5d6e279f4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:35.253822 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-40bb9caa-3c08-4b24-86d8-8c5d6e279f4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:36.180723 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-40bb9caa-3c08-4b24-86d8-8c5d6e279f4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:36.194486 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-40bb9caa-3c08-4b24-86d8-8c5d6e279f4b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1671 microversion: 2.1 time: 0.965456 Oct 24 09:01:36.195287 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 482/957] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:35 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1671 bytes in 967 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:01:36.898777 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-96bee6d7-64bb-4d96-99ae-b06b003eed73 req-68c79ce6-1a2c-43b9-b555-c29ec3ae8be9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5655b4d3-6c0f-4f8d-975d-024824e4a4b3", "name": "network-vif-deleted", "tag": "3cf51600-b21d-4989-92db-8e04b3392067"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:01:36.909996 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-96bee6d7-64bb-4d96-99ae-b06b003eed73 req-68c79ce6-1a2c-43b9-b555-c29ec3ae8be9 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:36.910811 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-96bee6d7-64bb-4d96-99ae-b06b003eed73 req-68c79ce6-1a2c-43b9-b555-c29ec3ae8be9 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:36.920936 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-96bee6d7-64bb-4d96-99ae-b06b003eed73 req-68c79ce6-1a2c-43b9-b555-c29ec3ae8be9 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:36.947292 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-96bee6d7-64bb-4d96-99ae-b06b003eed73 req-68c79ce6-1a2c-43b9-b555-c29ec3ae8be9 service nova] Creating event network-vif-deleted:3cf51600-b21d-4989-92db-8e04b3392067 for instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 on hv-cinder-maste Oct 24 09:01:36.954831 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-96bee6d7-64bb-4d96-99ae-b06b003eed73 req-68c79ce6-1a2c-43b9-b555-c29ec3ae8be9 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.451863 Oct 24 09:01:36.955932 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 476/958] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:01:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:01:37.223252 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-fce7ac7a-9171-4b88-8756-811ab8b24bad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:37.224495 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-fce7ac7a-9171-4b88-8756-811ab8b24bad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:37.235005 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fce7ac7a-9171-4b88-8756-811ab8b24bad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:37.237504 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fce7ac7a-9171-4b88-8756-811ab8b24bad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:37.238034 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fce7ac7a-9171-4b88-8756-811ab8b24bad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:37.238919 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fce7ac7a-9171-4b88-8756-811ab8b24bad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:37.239720 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fce7ac7a-9171-4b88-8756-811ab8b24bad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:37.240201 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fce7ac7a-9171-4b88-8756-811ab8b24bad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:37.328893 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fce7ac7a-9171-4b88-8756-811ab8b24bad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:37.330017 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fce7ac7a-9171-4b88-8756-811ab8b24bad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:37.330688 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fce7ac7a-9171-4b88-8756-811ab8b24bad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:37.331495 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-fce7ac7a-9171-4b88-8756-811ab8b24bad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lazy-loading 'fault' on Instance uuid 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 09:01:37.528579 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-fce7ac7a-9171-4b88-8756-811ab8b24bad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '94169e17266b4e6fad7e068a4c1384c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45be9d45ba4c4a2ca1ed053a7263dbee', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:37.536652 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-fce7ac7a-9171-4b88-8756-811ab8b24bad tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 200 len: 1327 microversion: 2.1 time: 0.317511 Oct 24 09:01:37.540738 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 483/959] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:37 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 1327 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:01:38.566820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a7fd7422-537b-4b64-8a57-ef507ee973dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:38.567525 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-a7fd7422-537b-4b64-8a57-ef507ee973dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:38.577277 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a7fd7422-537b-4b64-8a57-ef507ee973dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:38.578225 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a7fd7422-537b-4b64-8a57-ef507ee973dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:38.582978 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a7fd7422-537b-4b64-8a57-ef507ee973dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:38.583729 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a7fd7422-537b-4b64-8a57-ef507ee973dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:38.584228 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a7fd7422-537b-4b64-8a57-ef507ee973dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:38.584659 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a7fd7422-537b-4b64-8a57-ef507ee973dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:38.603302 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-a7fd7422-537b-4b64-8a57-ef507ee973dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] HTTP exception thrown: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a7fd7422-537b-4b64-8a57-ef507ee973dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Request method failure captured: Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 HTTP/1.1 Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlJb9biqZLl4tWim6-VtdE2wWG9xRCxMCbYJwKuM8w9jeHPhVjvEAVBfMCX4oubDLKjVnEHUr-ThOh3GW94chJTe-e8dcvllpAzphf2yg8VKeTt2KTF7-NusT3oCU-alk_qv54dC8z12P3bjjNWfhm3BLDPH-75ZWBuVhbBIUEzWS_8 Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 45be9d45ba4c4a2ca1ed053a7263dbee Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-AttachVolumeTestJSON-791554923 Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 09:01:38.605679 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 09:01:38.607447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/45be9d45ba4c4a2ca1ed053a7263dbee"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_45be9d45ba4c4a2ca1ed053a7263dbee", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/45be9d45ba4c4a2ca1ed053a7263dbee"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/45be9d45ba4c4a2ca1ed053a7263dbee"}]}] Oct 24 09:01:38.607447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-AttachVolumeTestJSON-791554923 Oct 24 09:01:38.607447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 45be9d45ba4c4a2ca1ed053a7263dbee Oct 24 09:01:38.607447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-AttachVolumeTestJSON-791554923 Oct 24 09:01:38.607447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-AttachVolumeTestJSON-791554923-project Oct 24 09:01:38.607447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 09:01:38.607447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 09:01:38.607447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 94169e17266b4e6fad7e068a4c1384c7 Oct 24 09:01:38.607447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-AttachVolumeTestJSON-791554923-project Oct 24 09:01:38.607447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 09:01:38.607447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. Oct 24 09:01:38.607447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': '5655b4d3-6c0f-4f8d-975d-024824e4a4b3'} Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:38.608450 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. Oct 24 09:01:38.610399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:38.613117 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a7fd7422-537b-4b64-8a57-ef507ee973dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Returning 404 to user: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:38.613117 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-a7fd7422-537b-4b64-8a57-ef507ee973dc tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 404 len: 111 microversion: 2.1 time: 0.045372 Oct 24 09:01:38.613117 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 477/960] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:38 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:01:38.621346 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f78af358-f6b4-4e7b-ad00-2ba322b6ff21 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '.version_select at 0x7fdfedc22310>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:38.648612 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f78af358-f6b4-4e7b-ad00-2ba322b6ff21 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1933428633" status: 202 len: 0 microversion: 2.1 time: 0.035300 Oct 24 09:01:38.649194 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 484/961] 10.222.0.39 () {58 vars in 1221 bytes} [Mon Oct 24 09:01:38 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1933428633 => generated 0 bytes in 37 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 09:01:39.967288 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-79e2f0c2-3788-4ee7-a926-97f7ddabf929 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:39.968980 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-79e2f0c2-3788-4ee7-a926-97f7ddabf929 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:39.998913 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-79e2f0c2-3788-4ee7-a926-97f7ddabf929 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:39.999425 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-79e2f0c2-3788-4ee7-a926-97f7ddabf929 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:39.999870 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-79e2f0c2-3788-4ee7-a926-97f7ddabf929 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:40.000633 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-79e2f0c2-3788-4ee7-a926-97f7ddabf929 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:40.004002 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-79e2f0c2-3788-4ee7-a926-97f7ddabf929 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:40.004870 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-79e2f0c2-3788-4ee7-a926-97f7ddabf929 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:40.133850 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.objects.instance [None req-79e2f0c2-3788-4ee7-a926-97f7ddabf929 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lazy-loading 'services' on Instance uuid 240dec37-c7ff-4c59-8b5f-611a52575853 {{(pid=130396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 09:01:40.255763 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-79e2f0c2-3788-4ee7-a926-97f7ddabf929 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "DELETE /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853/os-volume_attachments/3f302bb0-a01d-41ae-b11c-26bae986ef75" status: 202 len: 0 microversion: 2.1 time: 0.292966 Oct 24 09:01:40.267633 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-df62541b-f745-45ff-b223-04a31455e807 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:40.270322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-df62541b-f745-45ff-b223-04a31455e807 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:40.272542 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 478/962] 10.222.0.39 () {58 vars in 1416 bytes} [Mon Oct 24 09:01:39 2022] DELETE /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853/os-volume_attachments/3f302bb0-a01d-41ae-b11c-26bae986ef75 => generated 0 bytes in 310 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 09:01:40.286810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-df62541b-f745-45ff-b223-04a31455e807 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:40.287428 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-df62541b-f745-45ff-b223-04a31455e807 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:40.287854 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-df62541b-f745-45ff-b223-04a31455e807 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:40.288650 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-df62541b-f745-45ff-b223-04a31455e807 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:40.289944 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-df62541b-f745-45ff-b223-04a31455e807 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:40.291091 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-df62541b-f745-45ff-b223-04a31455e807 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:40.336600 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-df62541b-f745-45ff-b223-04a31455e807 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.073446 Oct 24 09:01:40.338435 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 485/963] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 09:01:40 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:01:41.355796 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-19c02fca-14c8-4ef6-8518-cec6ec655973 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:41.359524 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-19c02fca-14c8-4ef6-8518-cec6ec655973 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:41.371319 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-19c02fca-14c8-4ef6-8518-cec6ec655973 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:41.371972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-19c02fca-14c8-4ef6-8518-cec6ec655973 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:41.373386 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-19c02fca-14c8-4ef6-8518-cec6ec655973 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:41.374357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-19c02fca-14c8-4ef6-8518-cec6ec655973 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:41.374895 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-19c02fca-14c8-4ef6-8518-cec6ec655973 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:41.375563 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-19c02fca-14c8-4ef6-8518-cec6ec655973 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:41.453351 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-19c02fca-14c8-4ef6-8518-cec6ec655973 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.103231 Oct 24 09:01:41.458976 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 479/964] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 09:01:41 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853/os-volume_attachments => generated 197 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:01:42.008037 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-192bcccd-1420-43c5-a07d-debecb9cdd8e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:42.008638 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-192bcccd-1420-43c5-a07d-debecb9cdd8e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:42.016430 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-192bcccd-1420-43c5-a07d-debecb9cdd8e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:42.016928 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-192bcccd-1420-43c5-a07d-debecb9cdd8e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:42.017663 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-192bcccd-1420-43c5-a07d-debecb9cdd8e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:42.018406 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-192bcccd-1420-43c5-a07d-debecb9cdd8e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:42.018869 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-192bcccd-1420-43c5-a07d-debecb9cdd8e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:42.019300 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-192bcccd-1420-43c5-a07d-debecb9cdd8e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:42.037602 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-192bcccd-1420-43c5-a07d-debecb9cdd8e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] HTTP exception thrown: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-192bcccd-1420-43c5-a07d-debecb9cdd8e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Request method failure captured: Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: DELETE /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 HTTP/1.1 Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlJb9biqZLl4tWim6-VtdE2wWG9xRCxMCbYJwKuM8w9jeHPhVjvEAVBfMCX4oubDLKjVnEHUr-ThOh3GW94chJTe-e8dcvllpAzphf2yg8VKeTt2KTF7-NusT3oCU-alk_qv54dC8z12P3bjjNWfhm3BLDPH-75ZWBuVhbBIUEzWS_8 Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: 45be9d45ba4c4a2ca1ed053a7263dbee Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-AttachVolumeTestJSON-791554923 Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:01:42.039335 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:01:42.040740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/45be9d45ba4c4a2ca1ed053a7263dbee"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_45be9d45ba4c4a2ca1ed053a7263dbee", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/45be9d45ba4c4a2ca1ed053a7263dbee"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/45be9d45ba4c4a2ca1ed053a7263dbee"}]}] Oct 24 09:01:42.040740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-AttachVolumeTestJSON-791554923 Oct 24 09:01:42.040740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: 45be9d45ba4c4a2ca1ed053a7263dbee Oct 24 09:01:42.040740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-AttachVolumeTestJSON-791554923 Oct 24 09:01:42.040740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-AttachVolumeTestJSON-791554923-project Oct 24 09:01:42.040740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:01:42.040740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:01:42.040740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: 94169e17266b4e6fad7e068a4c1384c7 Oct 24 09:01:42.040740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-AttachVolumeTestJSON-791554923-project Oct 24 09:01:42.040740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 09:01:42.040740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. Oct 24 09:01:42.040740 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': '5655b4d3-6c0f-4f8d-975d-024824e4a4b3'} Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. Oct 24 09:01:42.043104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:42.045214 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-192bcccd-1420-43c5-a07d-debecb9cdd8e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Returning 404 to user: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:42.045214 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-192bcccd-1420-43c5-a07d-debecb9cdd8e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "DELETE /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 404 len: 111 microversion: 2.1 time: 0.036856 Oct 24 09:01:42.045214 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 486/965] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 09:01:42 2022] DELETE /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:01:42.051658 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-99337671-0c28-4d04-9b08-719d95d4019b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:42.052312 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-99337671-0c28-4d04-9b08-719d95d4019b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 5655b4d3-6c0f-4f8d-975d-024824e4a4b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:42.060093 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99337671-0c28-4d04-9b08-719d95d4019b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:42.060618 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99337671-0c28-4d04-9b08-719d95d4019b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:42.061056 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99337671-0c28-4d04-9b08-719d95d4019b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:42.061776 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99337671-0c28-4d04-9b08-719d95d4019b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:42.062230 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99337671-0c28-4d04-9b08-719d95d4019b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:42.062660 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-99337671-0c28-4d04-9b08-719d95d4019b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:42.078913 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-99337671-0c28-4d04-9b08-719d95d4019b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] HTTP exception thrown: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-99337671-0c28-4d04-9b08-719d95d4019b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Request method failure captured: Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 HTTP/1.1 Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlJb9biqZLl4tWim6-VtdE2wWG9xRCxMCbYJwKuM8w9jeHPhVjvEAVBfMCX4oubDLKjVnEHUr-ThOh3GW94chJTe-e8dcvllpAzphf2yg8VKeTt2KTF7-NusT3oCU-alk_qv54dC8z12P3bjjNWfhm3BLDPH-75ZWBuVhbBIUEzWS_8 Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 45be9d45ba4c4a2ca1ed053a7263dbee Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-AttachVolumeTestJSON-791554923 Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 09:01:42.080666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 09:01:42.082282 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/45be9d45ba4c4a2ca1ed053a7263dbee"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_45be9d45ba4c4a2ca1ed053a7263dbee", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/45be9d45ba4c4a2ca1ed053a7263dbee"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/45be9d45ba4c4a2ca1ed053a7263dbee"}]}] Oct 24 09:01:42.082282 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-AttachVolumeTestJSON-791554923 Oct 24 09:01:42.082282 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 45be9d45ba4c4a2ca1ed053a7263dbee Oct 24 09:01:42.082282 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-AttachVolumeTestJSON-791554923 Oct 24 09:01:42.082282 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-AttachVolumeTestJSON-791554923-project Oct 24 09:01:42.082282 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 09:01:42.082282 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 09:01:42.082282 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 94169e17266b4e6fad7e068a4c1384c7 Oct 24 09:01:42.082282 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-AttachVolumeTestJSON-791554923-project Oct 24 09:01:42.082282 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 09:01:42.082282 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. Oct 24 09:01:42.082282 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': '5655b4d3-6c0f-4f8d-975d-024824e4a4b3'} Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:42.083116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. Oct 24 09:01:42.084813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:42.086317 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-99337671-0c28-4d04-9b08-719d95d4019b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Returning 404 to user: Instance 5655b4d3-6c0f-4f8d-975d-024824e4a4b3 could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:42.086317 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-99337671-0c28-4d04-9b08-719d95d4019b tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3" status: 404 len: 111 microversion: 2.1 time: 0.034475 Oct 24 09:01:42.086317 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 480/966] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:42 2022] GET /compute/v2.1/servers/5655b4d3-6c0f-4f8d-975d-024824e4a4b3 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:01:42.466399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-308cdd31-faa0-442d-8cbe-d5c3dcf421ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:42.469656 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-308cdd31-faa0-442d-8cbe-d5c3dcf421ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:42.476632 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-308cdd31-faa0-442d-8cbe-d5c3dcf421ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:42.477245 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-308cdd31-faa0-442d-8cbe-d5c3dcf421ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:42.477775 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-308cdd31-faa0-442d-8cbe-d5c3dcf421ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:42.478749 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-308cdd31-faa0-442d-8cbe-d5c3dcf421ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:42.479900 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-308cdd31-faa0-442d-8cbe-d5c3dcf421ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:42.480601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-308cdd31-faa0-442d-8cbe-d5c3dcf421ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:42.533291 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-308cdd31-faa0-442d-8cbe-d5c3dcf421ef tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.070560 Oct 24 09:01:42.534025 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 487/967] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 09:01:42 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853/os-volume_attachments => generated 25 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 24 09:01:42.626529 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-69b298dd-43d0-41dc-9e72-e589799a245d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:42.627121 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-69b298dd-43d0-41dc-9e72-e589799a245d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:42.635624 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-69b298dd-43d0-41dc-9e72-e589799a245d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:42.636221 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-69b298dd-43d0-41dc-9e72-e589799a245d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:42.636662 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-69b298dd-43d0-41dc-9e72-e589799a245d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:42.637368 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-69b298dd-43d0-41dc-9e72-e589799a245d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:42.637874 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-69b298dd-43d0-41dc-9e72-e589799a245d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:42.638459 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-69b298dd-43d0-41dc-9e72-e589799a245d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:42.667291 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-69b298dd-43d0-41dc-9e72-e589799a245d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Going to try to terminate instance {{(pid=130396) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 09:01:42.735955 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:01:42.778214 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-69b298dd-43d0-41dc-9e72-e589799a245d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "DELETE /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 204 len: 0 microversion: 2.1 time: 0.155400 Oct 24 09:01:42.779206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 481/968] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 09:01:42 2022] DELETE /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 09:01:42.787842 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-46c3528d-debe-4b32-b49f-d0119ccce2a0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:42.790725 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-46c3528d-debe-4b32-b49f-d0119ccce2a0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:42.799125 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-46c3528d-debe-4b32-b49f-d0119ccce2a0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:42.799897 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-46c3528d-debe-4b32-b49f-d0119ccce2a0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:42.800374 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-46c3528d-debe-4b32-b49f-d0119ccce2a0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:42.801114 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-46c3528d-debe-4b32-b49f-d0119ccce2a0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:42.801576 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-46c3528d-debe-4b32-b49f-d0119ccce2a0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:42.802058 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-46c3528d-debe-4b32-b49f-d0119ccce2a0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:42.988182 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-46c3528d-debe-4b32-b49f-d0119ccce2a0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:42.998713 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-46c3528d-debe-4b32-b49f-d0119ccce2a0 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1679 microversion: 2.1 time: 0.215025 Oct 24 09:01:43.016197 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 488/969] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:42 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1679 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:01:43.294491 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2dc38069-5438-46eb-bcac-3308896c872e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:43.295480 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2dc38069-5438-46eb-bcac-3308896c872e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:43.303452 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2dc38069-5438-46eb-bcac-3308896c872e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:43.304237 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2dc38069-5438-46eb-bcac-3308896c872e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:43.304915 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2dc38069-5438-46eb-bcac-3308896c872e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:43.308049 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2dc38069-5438-46eb-bcac-3308896c872e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:43.308796 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2dc38069-5438-46eb-bcac-3308896c872e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:43.309335 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2dc38069-5438-46eb-bcac-3308896c872e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:43.329847 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-2dc38069-5438-46eb-bcac-3308896c872e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] HTTP exception thrown: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2dc38069-5438-46eb-bcac-3308896c872e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Request method failure captured: Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: DELETE /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 HTTP/1.1 Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlJb9biqZLl4tWim6-VtdE2wWG9xRCxMCbYJwKuM8w9jeHPhVjvEAVBfMCX4oubDLKjVnEHUr-ThOh3GW94chJTe-e8dcvllpAzphf2yg8VKeTt2KTF7-NusT3oCU-alk_qv54dC8z12P3bjjNWfhm3BLDPH-75ZWBuVhbBIUEzWS_8 Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 45be9d45ba4c4a2ca1ed053a7263dbee Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-AttachVolumeTestJSON-791554923 Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 09:01:43.331403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 09:01:43.332820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/45be9d45ba4c4a2ca1ed053a7263dbee"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_45be9d45ba4c4a2ca1ed053a7263dbee", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/45be9d45ba4c4a2ca1ed053a7263dbee"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/45be9d45ba4c4a2ca1ed053a7263dbee"}]}] Oct 24 09:01:43.332820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-AttachVolumeTestJSON-791554923 Oct 24 09:01:43.332820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 45be9d45ba4c4a2ca1ed053a7263dbee Oct 24 09:01:43.332820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-AttachVolumeTestJSON-791554923 Oct 24 09:01:43.332820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-AttachVolumeTestJSON-791554923-project Oct 24 09:01:43.332820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 09:01:43.332820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 09:01:43.332820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 94169e17266b4e6fad7e068a4c1384c7 Oct 24 09:01:43.332820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-AttachVolumeTestJSON-791554923-project Oct 24 09:01:43.332820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 09:01:43.332820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. Oct 24 09:01:43.332820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': '89a2295a-90e9-4fc1-be3b-a130f7e71528'} Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:43.333610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. Oct 24 09:01:43.334890 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:43.337622 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2dc38069-5438-46eb-bcac-3308896c872e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Returning 404 to user: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:43.339818 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2dc38069-5438-46eb-bcac-3308896c872e tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "DELETE /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 404 len: 111 microversion: 2.1 time: 0.049520 Oct 24 09:01:43.343289 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 482/970] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 09:01:43 2022] DELETE /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:01:43.348547 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ca0b9a9c-d2e0-4ffc-b31b-21ba1b35f8f8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:43.349539 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ca0b9a9c-d2e0-4ffc-b31b-21ba1b35f8f8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] [instance: 89a2295a-90e9-4fc1-be3b-a130f7e71528] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:43.357113 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ca0b9a9c-d2e0-4ffc-b31b-21ba1b35f8f8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:43.357914 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ca0b9a9c-d2e0-4ffc-b31b-21ba1b35f8f8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:43.358581 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ca0b9a9c-d2e0-4ffc-b31b-21ba1b35f8f8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:43.359521 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ca0b9a9c-d2e0-4ffc-b31b-21ba1b35f8f8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:43.360252 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ca0b9a9c-d2e0-4ffc-b31b-21ba1b35f8f8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:43.360981 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ca0b9a9c-d2e0-4ffc-b31b-21ba1b35f8f8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:43.376898 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-ca0b9a9c-d2e0-4ffc-b31b-21ba1b35f8f8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] HTTP exception thrown: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ca0b9a9c-d2e0-4ffc-b31b-21ba1b35f8f8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Request method failure captured: Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 HTTP/1.1 Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlJb9biqZLl4tWim6-VtdE2wWG9xRCxMCbYJwKuM8w9jeHPhVjvEAVBfMCX4oubDLKjVnEHUr-ThOh3GW94chJTe-e8dcvllpAzphf2yg8VKeTt2KTF7-NusT3oCU-alk_qv54dC8z12P3bjjNWfhm3BLDPH-75ZWBuVhbBIUEzWS_8 Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: 45be9d45ba4c4a2ca1ed053a7263dbee Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-AttachVolumeTestJSON-791554923 Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:01:43.382292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:01:43.383698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/45be9d45ba4c4a2ca1ed053a7263dbee"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_45be9d45ba4c4a2ca1ed053a7263dbee", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/45be9d45ba4c4a2ca1ed053a7263dbee"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/45be9d45ba4c4a2ca1ed053a7263dbee"}]}] Oct 24 09:01:43.383698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-AttachVolumeTestJSON-791554923 Oct 24 09:01:43.383698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: 45be9d45ba4c4a2ca1ed053a7263dbee Oct 24 09:01:43.383698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-AttachVolumeTestJSON-791554923 Oct 24 09:01:43.383698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-AttachVolumeTestJSON-791554923-project Oct 24 09:01:43.383698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:01:43.383698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:01:43.383698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: 94169e17266b4e6fad7e068a4c1384c7 Oct 24 09:01:43.383698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-AttachVolumeTestJSON-791554923-project Oct 24 09:01:43.383698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 09:01:43.383698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. Oct 24 09:01:43.383698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': '89a2295a-90e9-4fc1-be3b-a130f7e71528'} Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:43.384587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. Oct 24 09:01:43.386230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:43.389655 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ca0b9a9c-d2e0-4ffc-b31b-21ba1b35f8f8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] Returning 404 to user: Instance 89a2295a-90e9-4fc1-be3b-a130f7e71528 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:43.390924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ca0b9a9c-d2e0-4ffc-b31b-21ba1b35f8f8 tempest-AttachVolumeTestJSON-791554923 tempest-AttachVolumeTestJSON-791554923-project] 10.222.0.39 "GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528" status: 404 len: 111 microversion: 2.1 time: 0.046174 Oct 24 09:01:43.393039 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 489/971] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:43 2022] GET /compute/v2.1/servers/89a2295a-90e9-4fc1-be3b-a130f7e71528 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:01:44.023057 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-638c4eea-ed1d-4df1-884e-d7b47d528353 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:44.023677 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-638c4eea-ed1d-4df1-884e-d7b47d528353 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:44.034870 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-638c4eea-ed1d-4df1-884e-d7b47d528353 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:44.035357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-638c4eea-ed1d-4df1-884e-d7b47d528353 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:44.036641 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-638c4eea-ed1d-4df1-884e-d7b47d528353 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:44.038304 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-638c4eea-ed1d-4df1-884e-d7b47d528353 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:44.038925 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-638c4eea-ed1d-4df1-884e-d7b47d528353 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:44.039511 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-638c4eea-ed1d-4df1-884e-d7b47d528353 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:44.355030 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-638c4eea-ed1d-4df1-884e-d7b47d528353 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:44.366361 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-638c4eea-ed1d-4df1-884e-d7b47d528353 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1679 microversion: 2.1 time: 0.347260 Oct 24 09:01:44.372965 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 483/972] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:44 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1679 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:01:45.395024 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ee084c04-06f5-4d16-9bf8-e4eff8e1637d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:45.396409 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ee084c04-06f5-4d16-9bf8-e4eff8e1637d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:45.409533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ee084c04-06f5-4d16-9bf8-e4eff8e1637d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:45.410473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ee084c04-06f5-4d16-9bf8-e4eff8e1637d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:45.411138 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ee084c04-06f5-4d16-9bf8-e4eff8e1637d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:45.412171 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ee084c04-06f5-4d16-9bf8-e4eff8e1637d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:45.412850 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ee084c04-06f5-4d16-9bf8-e4eff8e1637d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:45.414062 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ee084c04-06f5-4d16-9bf8-e4eff8e1637d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:46.054005 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ee084c04-06f5-4d16-9bf8-e4eff8e1637d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:46.058293 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=3,nova_cell1:UPDATE=2 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:01:46.067923 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ee084c04-06f5-4d16-9bf8-e4eff8e1637d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1679 microversion: 2.1 time: 0.676841 Oct 24 09:01:46.080640 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 490/973] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:45 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1679 bytes in 682 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:01:46.297095 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-21b38571-b9f0-41c8-b857-6a546efd6d9e req-d97a889b-baae-4534-b397-4433a3a0ad01 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "240dec37-c7ff-4c59-8b5f-611a52575853", "name": "network-vif-deleted", "tag": "86414159-5cb9-488f-8380-698e8b8e4c32"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:01:46.322218 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-21b38571-b9f0-41c8-b857-6a546efd6d9e req-d97a889b-baae-4534-b397-4433a3a0ad01 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:46.323030 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-21b38571-b9f0-41c8-b857-6a546efd6d9e req-d97a889b-baae-4534-b397-4433a3a0ad01 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:46.324218 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-21b38571-b9f0-41c8-b857-6a546efd6d9e req-d97a889b-baae-4534-b397-4433a3a0ad01 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:46.341227 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-21b38571-b9f0-41c8-b857-6a546efd6d9e req-d97a889b-baae-4534-b397-4433a3a0ad01 service nova] Creating event network-vif-deleted:86414159-5cb9-488f-8380-698e8b8e4c32 for instance 240dec37-c7ff-4c59-8b5f-611a52575853 on hv-cinder-maste Oct 24 09:01:46.351201 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-21b38571-b9f0-41c8-b857-6a546efd6d9e req-d97a889b-baae-4534-b397-4433a3a0ad01 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058532 Oct 24 09:01:46.352069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 484/974] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:01:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:01:47.099437 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-437d47a1-e2da-429d-8d19-aa2f652ed98d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:47.100842 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-437d47a1-e2da-429d-8d19-aa2f652ed98d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:47.119691 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-437d47a1-e2da-429d-8d19-aa2f652ed98d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:47.120892 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-437d47a1-e2da-429d-8d19-aa2f652ed98d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:47.121535 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-437d47a1-e2da-429d-8d19-aa2f652ed98d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:47.122649 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-437d47a1-e2da-429d-8d19-aa2f652ed98d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:47.131513 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-437d47a1-e2da-429d-8d19-aa2f652ed98d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:47.132300 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-437d47a1-e2da-429d-8d19-aa2f652ed98d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:47.248575 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-437d47a1-e2da-429d-8d19-aa2f652ed98d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:47.252008 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-437d47a1-e2da-429d-8d19-aa2f652ed98d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:47.252658 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-437d47a1-e2da-429d-8d19-aa2f652ed98d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:47.253588 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-437d47a1-e2da-429d-8d19-aa2f652ed98d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lazy-loading 'fault' on Instance uuid 240dec37-c7ff-4c59-8b5f-611a52575853 {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 09:01:47.419676 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-437d47a1-e2da-429d-8d19-aa2f652ed98d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ebdef1d69bb14837b9a0c28cbba332fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b34f10ec378a4622a8f0b0546e84da7f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:47.429364 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-437d47a1-e2da-429d-8d19-aa2f652ed98d tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 200 len: 1330 microversion: 2.1 time: 0.336380 Oct 24 09:01:47.431171 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 491/975] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:47 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 1330 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:01:47.954712 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b06b2461-9967-484f-ac19-659c7fa88c3c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:47.957756 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b06b2461-9967-484f-ac19-659c7fa88c3c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:47.960582 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=45,nova_api:UPDATE=2 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:01:47.972291 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b06b2461-9967-484f-ac19-659c7fa88c3c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:47.975287 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b06b2461-9967-484f-ac19-659c7fa88c3c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:47.976234 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b06b2461-9967-484f-ac19-659c7fa88c3c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:47.977191 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b06b2461-9967-484f-ac19-659c7fa88c3c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:47.981424 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b06b2461-9967-484f-ac19-659c7fa88c3c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:47.982739 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b06b2461-9967-484f-ac19-659c7fa88c3c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:48.022660 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b06b2461-9967-484f-ac19-659c7fa88c3c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.076244 Oct 24 09:01:48.023834 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 485/976] 10.222.0.39 () {60 vars in 1343 bytes} [Mon Oct 24 09:01:47 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3/os-volume_attachments => generated 25 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:01:48.117708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-43b8342d-6b9d-4d39-bbf9-82b6fbee7f17 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:48.118535 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-43b8342d-6b9d-4d39-bbf9-82b6fbee7f17 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:48.121844 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=40,nova_api:DELETE=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:01:48.133500 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-43b8342d-6b9d-4d39-bbf9-82b6fbee7f17 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:48.134087 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-43b8342d-6b9d-4d39-bbf9-82b6fbee7f17 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:48.134527 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-43b8342d-6b9d-4d39-bbf9-82b6fbee7f17 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:48.135242 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-43b8342d-6b9d-4d39-bbf9-82b6fbee7f17 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:48.135689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-43b8342d-6b9d-4d39-bbf9-82b6fbee7f17 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:48.144481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-43b8342d-6b9d-4d39-bbf9-82b6fbee7f17 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:48.187793 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-43b8342d-6b9d-4d39-bbf9-82b6fbee7f17 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Going to try to terminate instance {{(pid=130395) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 09:01:48.356634 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-43b8342d-6b9d-4d39-bbf9-82b6fbee7f17 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "DELETE /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 204 len: 0 microversion: 2.20 time: 0.233279 Oct 24 09:01:48.357835 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 492/977] 10.222.0.39 () {60 vars in 1280 bytes} [Mon Oct 24 09:01:48 2022] DELETE /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 24 09:01:48.367716 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c68896ce-f63f-451b-961b-65e16eba815b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:48.368589 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c68896ce-f63f-451b-961b-65e16eba815b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:48.376410 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c68896ce-f63f-451b-961b-65e16eba815b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:48.377343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c68896ce-f63f-451b-961b-65e16eba815b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:48.378042 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c68896ce-f63f-451b-961b-65e16eba815b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:48.379078 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c68896ce-f63f-451b-961b-65e16eba815b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:48.379800 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c68896ce-f63f-451b-961b-65e16eba815b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:48.380532 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c68896ce-f63f-451b-961b-65e16eba815b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:48.457655 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b85f5c44-b193-4194-b8f1-ef56b618f2a6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:48.458632 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b85f5c44-b193-4194-b8f1-ef56b618f2a6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:48.470739 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b85f5c44-b193-4194-b8f1-ef56b618f2a6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:48.471816 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b85f5c44-b193-4194-b8f1-ef56b618f2a6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:48.474267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b85f5c44-b193-4194-b8f1-ef56b618f2a6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:48.475248 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b85f5c44-b193-4194-b8f1-ef56b618f2a6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:48.482203 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b85f5c44-b193-4194-b8f1-ef56b618f2a6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:48.483024 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b85f5c44-b193-4194-b8f1-ef56b618f2a6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:48.524377 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-b85f5c44-b193-4194-b8f1-ef56b618f2a6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] HTTP exception thrown: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b85f5c44-b193-4194-b8f1-ef56b618f2a6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Request method failure captured: Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 HTTP/1.1 Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlH_P2cgqdGxTyt_DjhRKpesrNF98h-yweC7dRlZggOYZXXonk3ROJmezNObJqlbezl2V7Pqsl8fDKOVcqEGBVvIZd3l8-ZBbHliak_eivuyadmAWxfis-R_etsiC3KVVo1LvGlKxFu0GFkQBrTFC5gIfs64iGB92PfDe2tNtbCL85U Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:01:48.526080 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:01:48.527960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b34f10ec378a4622a8f0b0546e84da7f", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}] Oct 24 09:01:48.527960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:48.527960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:48.527960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:48.527960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:48.527960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:01:48.527960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:01:48.527960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: ebdef1d69bb14837b9a0c28cbba332fc Oct 24 09:01:48.527960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:48.527960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 09:01:48.527960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. Oct 24 09:01:48.527960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': '240dec37-c7ff-4c59-8b5f-611a52575853'} Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:48.528821 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. Oct 24 09:01:48.530167 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:48.531352 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b85f5c44-b193-4194-b8f1-ef56b618f2a6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Returning 404 to user: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:48.531352 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b85f5c44-b193-4194-b8f1-ef56b618f2a6 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 404 len: 111 microversion: 2.1 time: 0.074169 Oct 24 09:01:48.531352 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 493/978] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:48 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 111 bytes in 76 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 24 09:01:48.538983 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-69caa14d-9c01-4e97-8fcd-b6f08d5954f9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '.version_select at 0x7fdfeded3160>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:48.581988 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-69caa14d-9c01-4e97-8fcd-b6f08d5954f9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-186264620" status: 202 len: 0 microversion: 2.1 time: 0.049240 Oct 24 09:01:48.601105 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 494/979] 10.222.0.39 () {58 vars in 1218 bytes} [Mon Oct 24 09:01:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-186264620 => generated 0 bytes in 68 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 24 09:01:48.747340 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c68896ce-f63f-451b-961b-65e16eba815b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:48.781886 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c68896ce-f63f-451b-961b-65e16eba815b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:48.783735 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c68896ce-f63f-451b-961b-65e16eba815b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:48.785416 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c68896ce-f63f-451b-961b-65e16eba815b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1723 microversion: 2.20 time: 0.423314 Oct 24 09:01:48.786878 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 486/980] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:01:48 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1723 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 09:01:49.818642 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-83820972-ec2e-495e-a18d-2cacf4b54ed4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:49.819363 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-83820972-ec2e-495e-a18d-2cacf4b54ed4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:49.839580 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-83820972-ec2e-495e-a18d-2cacf4b54ed4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:49.847204 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-83820972-ec2e-495e-a18d-2cacf4b54ed4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:49.848659 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-83820972-ec2e-495e-a18d-2cacf4b54ed4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:49.849544 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-83820972-ec2e-495e-a18d-2cacf4b54ed4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:49.850114 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-83820972-ec2e-495e-a18d-2cacf4b54ed4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:49.850650 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-83820972-ec2e-495e-a18d-2cacf4b54ed4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:50.107771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-83820972-ec2e-495e-a18d-2cacf4b54ed4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:50.118050 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-83820972-ec2e-495e-a18d-2cacf4b54ed4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:50.119515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-83820972-ec2e-495e-a18d-2cacf4b54ed4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:50.122285 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-83820972-ec2e-495e-a18d-2cacf4b54ed4 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1723 microversion: 2.20 time: 0.308465 Oct 24 09:01:50.123372 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 495/981] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:01:49 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1723 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 09:01:51.000926 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a4992a3f-663e-41e6-9a6e-0bdcc8ccc17b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:51.001669 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-a4992a3f-663e-41e6-9a6e-0bdcc8ccc17b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:51.009493 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a4992a3f-663e-41e6-9a6e-0bdcc8ccc17b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:51.009813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a4992a3f-663e-41e6-9a6e-0bdcc8ccc17b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:51.010249 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a4992a3f-663e-41e6-9a6e-0bdcc8ccc17b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:51.011877 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a4992a3f-663e-41e6-9a6e-0bdcc8ccc17b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:51.012610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a4992a3f-663e-41e6-9a6e-0bdcc8ccc17b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:51.013063 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a4992a3f-663e-41e6-9a6e-0bdcc8ccc17b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:51.030342 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-a4992a3f-663e-41e6-9a6e-0bdcc8ccc17b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] HTTP exception thrown: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a4992a3f-663e-41e6-9a6e-0bdcc8ccc17b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Request method failure captured: Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: DELETE /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 HTTP/1.1 Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlH_P2cgqdGxTyt_DjhRKpesrNF98h-yweC7dRlZggOYZXXonk3ROJmezNObJqlbezl2V7Pqsl8fDKOVcqEGBVvIZd3l8-ZBbHliak_eivuyadmAWxfis-R_etsiC3KVVo1LvGlKxFu0GFkQBrTFC5gIfs64iGB92PfDe2tNtbCL85U Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 09:01:51.032151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 09:01:51.033701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b34f10ec378a4622a8f0b0546e84da7f", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}] Oct 24 09:01:51.033701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:51.033701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:51.033701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:51.033701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:51.033701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 09:01:51.033701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 09:01:51.033701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: ebdef1d69bb14837b9a0c28cbba332fc Oct 24 09:01:51.033701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:51.033701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 09:01:51.033701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. Oct 24 09:01:51.033701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': '240dec37-c7ff-4c59-8b5f-611a52575853'} Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:51.036732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. Oct 24 09:01:51.038118 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:51.039558 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a4992a3f-663e-41e6-9a6e-0bdcc8ccc17b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Returning 404 to user: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:51.039558 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-a4992a3f-663e-41e6-9a6e-0bdcc8ccc17b tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "DELETE /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 404 len: 111 microversion: 2.1 time: 0.039424 Oct 24 09:01:51.039558 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 487/982] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 09:01:50 2022] DELETE /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:01:51.042398 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1ac1c213-e569-4292-a954-ecf5f4e4a505 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:51.043218 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1ac1c213-e569-4292-a954-ecf5f4e4a505 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 240dec37-c7ff-4c59-8b5f-611a52575853] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:51.055385 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ac1c213-e569-4292-a954-ecf5f4e4a505 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:51.055885 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ac1c213-e569-4292-a954-ecf5f4e4a505 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:51.056391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ac1c213-e569-4292-a954-ecf5f4e4a505 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:51.057195 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ac1c213-e569-4292-a954-ecf5f4e4a505 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:51.057530 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ac1c213-e569-4292-a954-ecf5f4e4a505 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:51.058513 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1ac1c213-e569-4292-a954-ecf5f4e4a505 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:51.080157 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-1ac1c213-e569-4292-a954-ecf5f4e4a505 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] HTTP exception thrown: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1ac1c213-e569-4292-a954-ecf5f4e4a505 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Request method failure captured: Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 HTTP/1.1 Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlH_P2cgqdGxTyt_DjhRKpesrNF98h-yweC7dRlZggOYZXXonk3ROJmezNObJqlbezl2V7Pqsl8fDKOVcqEGBVvIZd3l8-ZBbHliak_eivuyadmAWxfis-R_etsiC3KVVo1LvGlKxFu0GFkQBrTFC5gIfs64iGB92PfDe2tNtbCL85U Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:01:51.080706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:01:51.082355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b34f10ec378a4622a8f0b0546e84da7f", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}] Oct 24 09:01:51.082355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:51.082355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:51.082355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:51.082355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:51.082355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:01:51.082355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:01:51.082355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: ebdef1d69bb14837b9a0c28cbba332fc Oct 24 09:01:51.082355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:51.082355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 09:01:51.082355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. Oct 24 09:01:51.082355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': '240dec37-c7ff-4c59-8b5f-611a52575853'} Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:51.086533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. Oct 24 09:01:51.088120 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:51.090138 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1ac1c213-e569-4292-a954-ecf5f4e4a505 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Returning 404 to user: Instance 240dec37-c7ff-4c59-8b5f-611a52575853 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:51.090138 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1ac1c213-e569-4292-a954-ecf5f4e4a505 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853" status: 404 len: 111 microversion: 2.1 time: 0.043643 Oct 24 09:01:51.090138 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 496/983] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:51 2022] GET /compute/v2.1/servers/240dec37-c7ff-4c59-8b5f-611a52575853 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:01:51.160680 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-789b7488-3cf1-4fe8-add0-f85648e6b17c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:51.161474 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-789b7488-3cf1-4fe8-add0-f85648e6b17c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:51.171989 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-789b7488-3cf1-4fe8-add0-f85648e6b17c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:51.172607 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-789b7488-3cf1-4fe8-add0-f85648e6b17c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:51.173087 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-789b7488-3cf1-4fe8-add0-f85648e6b17c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:51.173810 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-789b7488-3cf1-4fe8-add0-f85648e6b17c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:51.174297 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-789b7488-3cf1-4fe8-add0-f85648e6b17c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:51.174760 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-789b7488-3cf1-4fe8-add0-f85648e6b17c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:51.226952 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-e4bd6d85-ad33-4118-ba53-218e293d53b4 req-bb7f02ea-6ddc-4e60-9bd6-e70ae3b3a14e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e72acc7a-decb-46c7-9c51-f731e2c413b3", "name": "network-vif-deleted", "tag": "d0e79b5c-b764-405d-8c7a-d5032e3c72a8"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:01:51.264380 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-e4bd6d85-ad33-4118-ba53-218e293d53b4 req-bb7f02ea-6ddc-4e60-9bd6-e70ae3b3a14e service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:51.266107 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-e4bd6d85-ad33-4118-ba53-218e293d53b4 req-bb7f02ea-6ddc-4e60-9bd6-e70ae3b3a14e service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:51.269039 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-e4bd6d85-ad33-4118-ba53-218e293d53b4 req-bb7f02ea-6ddc-4e60-9bd6-e70ae3b3a14e service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:51.309632 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-e4bd6d85-ad33-4118-ba53-218e293d53b4 req-bb7f02ea-6ddc-4e60-9bd6-e70ae3b3a14e service nova] Creating event network-vif-deleted:d0e79b5c-b764-405d-8c7a-d5032e3c72a8 for instance e72acc7a-decb-46c7-9c51-f731e2c413b3 on hv-cinder-maste Oct 24 09:01:51.324353 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-789b7488-3cf1-4fe8-add0-f85648e6b17c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:51.333712 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-e4bd6d85-ad33-4118-ba53-218e293d53b4 req-bb7f02ea-6ddc-4e60-9bd6-e70ae3b3a14e service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.096871 Oct 24 09:01:51.337738 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 497/984] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:01:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:01:51.340148 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-789b7488-3cf1-4fe8-add0-f85648e6b17c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:51.342164 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-789b7488-3cf1-4fe8-add0-f85648e6b17c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '552c23b8456a40ec8e01297aca7965d3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7ff48b26c576495f8bb8c88d20d246cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:51.343721 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-789b7488-3cf1-4fe8-add0-f85648e6b17c tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 200 len: 1655 microversion: 2.20 time: 0.192448 Oct 24 09:01:51.344757 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 488/985] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:01:51 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 1655 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 24 09:01:52.033458 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-6e58b16f-da6e-4a04-bb59-7f583021b876 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Action: 'create', calling method: .version_select at 0x7fdfedd96a60>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-624156899", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJquRMCgF2eoNOPLKV11jNPmBVp4gcB7SXI+iOTbABdr+W4E6yRO/DmorPnfbmqLPZB4q9Q/3i4MM8CbF1E/7pM5F3enlx7kQikiWadfiO8MlK6+LRVSpsI2scijz5tAPQ=="}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:01:52.051325 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-6e58b16f-da6e-4a04-bb59-7f583021b876 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Getting quotas for project 7e9c4dec37674f73912f0d9272ac2446. Resources: {'key_pairs'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 09:01:52.055207 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-6e58b16f-da6e-4a04-bb59-7f583021b876 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Getting quotas for user 4f104bf97764426cb67b14bbf39e16be and project 7e9c4dec37674f73912f0d9272ac2446. Resources: {'key_pairs'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 09:01:52.084578 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-6e58b16f-da6e-4a04-bb59-7f583021b876 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.055421 Oct 24 09:01:52.088570 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 498/986] 10.222.0.39 () {60 vars in 1159 bytes} [Mon Oct 24 09:01:52 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:01:52.194221 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-2138141721", "imageRef": "b45a827e-4d67-4f58-abbe-bd39c4cf9586", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-624156899", "networks": [{"uuid": "fa6e8a64-0da3-4aac-b1d9-f74c3aa58bb7"}]}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:01:52.261491 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.network.neutron [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] validate_networks() for [('fa6e8a64-0da3-4aac-b1d9-f74c3aa58bb7', None, None, None, None, None)] {{(pid=130396) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 09:01:52.343278 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-dbd1ea8b-5dcf-4e23-91db-d0d3f666c43c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:52.343891 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-dbd1ea8b-5dcf-4e23-91db-d0d3f666c43c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:52.351289 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dbd1ea8b-5dcf-4e23-91db-d0d3f666c43c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:52.352060 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dbd1ea8b-5dcf-4e23-91db-d0d3f666c43c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:52.352697 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dbd1ea8b-5dcf-4e23-91db-d0d3f666c43c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:52.353564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dbd1ea8b-5dcf-4e23-91db-d0d3f666c43c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:52.354160 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dbd1ea8b-5dcf-4e23-91db-d0d3f666c43c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:52.355658 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dbd1ea8b-5dcf-4e23-91db-d0d3f666c43c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:52.386034 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-dbd1ea8b-5dcf-4e23-91db-d0d3f666c43c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] HTTP exception thrown: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-dbd1ea8b-5dcf-4e23-91db-d0d3f666c43c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Request method failure captured: Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: DELETE /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf HTTP/1.1 Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlH_P2cgqdGxTyt_DjhRKpesrNF98h-yweC7dRlZggOYZXXonk3ROJmezNObJqlbezl2V7Pqsl8fDKOVcqEGBVvIZd3l8-ZBbHliak_eivuyadmAWxfis-R_etsiC3KVVo1LvGlKxFu0GFkQBrTFC5gIfs64iGB92PfDe2tNtbCL85U Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:01:52.388047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:01:52.389692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b34f10ec378a4622a8f0b0546e84da7f", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}] Oct 24 09:01:52.389692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:52.389692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:52.389692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:52.389692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:52.389692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:01:52.389692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:01:52.389692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: ebdef1d69bb14837b9a0c28cbba332fc Oct 24 09:01:52.389692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:52.389692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 09:01:52.389692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. Oct 24 09:01:52.389692 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': 'd9301ca5-bdaa-4f5d-8bcd-772b8923b1cf'} Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:52.390601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. Oct 24 09:01:52.392047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.394552 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-dbd1ea8b-5dcf-4e23-91db-d0d3f666c43c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Returning 404 to user: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:52.395689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-dbd1ea8b-5dcf-4e23-91db-d0d3f666c43c tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "DELETE /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 404 len: 111 microversion: 2.1 time: 0.057477 Oct 24 09:01:52.400750 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 499/987] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 09:01:52 2022] DELETE /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:01:52.407693 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-313142fe-9ce0-4496-b6d5-93cab5b24d46 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:52.408576 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-313142fe-9ce0-4496-b6d5-93cab5b24d46 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:52.419126 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-313142fe-9ce0-4496-b6d5-93cab5b24d46 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:52.419617 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-313142fe-9ce0-4496-b6d5-93cab5b24d46 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:52.420056 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-313142fe-9ce0-4496-b6d5-93cab5b24d46 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:52.420900 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-313142fe-9ce0-4496-b6d5-93cab5b24d46 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:52.421249 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-313142fe-9ce0-4496-b6d5-93cab5b24d46 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:52.421676 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-313142fe-9ce0-4496-b6d5-93cab5b24d46 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:52.439730 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-313142fe-9ce0-4496-b6d5-93cab5b24d46 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] HTTP exception thrown: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-313142fe-9ce0-4496-b6d5-93cab5b24d46 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Request method failure captured: Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 HTTP/1.1 Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlH7mBC-BQLg0H8yK7G2g-HOPUEsdjQL5a9CNStTzz2vRIXjmn3BKvY3yRoT_VlpqjhmiOscl52jQCwEMfyH6tF1HUFCQtCOEAYK8cH5PiF_zTNaUGNGpBjSQoGhunfKNz-_3ckGEvv273V4IyJwd9U0Vvic8VBlr37q6YrJuFn9p-Y Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Openstack-Nova-Api-Version: 2.20 Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: 7ff48b26c576495f8bb8c88d20d246cc Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:01:52.441629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:01:52.443322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/7ff48b26c576495f8bb8c88d20d246cc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_7ff48b26c576495f8bb8c88d20d246cc", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/7ff48b26c576495f8bb8c88d20d246cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/7ff48b26c576495f8bb8c88d20d246cc"}]}] Oct 24 09:01:52.443322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 09:01:52.443322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: 7ff48b26c576495f8bb8c88d20d246cc Oct 24 09:01:52.443322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 09:01:52.443322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-AttachVolumeShelveTestJSON-160255499-project Oct 24 09:01:52.443322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:01:52.443322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:01:52.443322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: 552c23b8456a40ec8e01297aca7965d3 Oct 24 09:01:52.443322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-AttachVolumeShelveTestJSON-160255499-project Oct 24 09:01:52.443322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 09:01:52.443322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. Oct 24 09:01:52.443322 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': 'e72acc7a-decb-46c7-9c51-f731e2c413b3'} Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:52.444151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. Oct 24 09:01:52.445411 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.446584 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-313142fe-9ce0-4496-b6d5-93cab5b24d46 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Returning 404 to user: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:52.446584 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-313142fe-9ce0-4496-b6d5-93cab5b24d46 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 404 len: 111 microversion: 2.20 time: 0.040421 Oct 24 09:01:52.446584 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 500/988] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:01:52 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 24 09:01:52.450798 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-fbf62387-8874-4564-ae36-aceff6b3c1a3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:52.451414 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-fbf62387-8874-4564-ae36-aceff6b3c1a3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:52.460531 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fbf62387-8874-4564-ae36-aceff6b3c1a3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:52.461085 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fbf62387-8874-4564-ae36-aceff6b3c1a3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:52.461533 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fbf62387-8874-4564-ae36-aceff6b3c1a3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:52.462259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fbf62387-8874-4564-ae36-aceff6b3c1a3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:52.462720 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fbf62387-8874-4564-ae36-aceff6b3c1a3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:52.463143 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fbf62387-8874-4564-ae36-aceff6b3c1a3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:52.482495 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-fbf62387-8874-4564-ae36-aceff6b3c1a3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] HTTP exception thrown: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-fbf62387-8874-4564-ae36-aceff6b3c1a3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Request method failure captured: Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf HTTP/1.1 Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlH_P2cgqdGxTyt_DjhRKpesrNF98h-yweC7dRlZggOYZXXonk3ROJmezNObJqlbezl2V7Pqsl8fDKOVcqEGBVvIZd3l8-ZBbHliak_eivuyadmAWxfis-R_etsiC3KVVo1LvGlKxFu0GFkQBrTFC5gIfs64iGB92PfDe2tNtbCL85U Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:01:52.484399 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:01:52.486030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b34f10ec378a4622a8f0b0546e84da7f", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}] Oct 24 09:01:52.486030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:52.486030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:52.486030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:52.486030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:52.486030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:01:52.486030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:01:52.486030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: ebdef1d69bb14837b9a0c28cbba332fc Oct 24 09:01:52.486030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:52.486030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 09:01:52.486030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. Oct 24 09:01:52.486030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': 'd9301ca5-bdaa-4f5d-8bcd-772b8923b1cf'} Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:52.487013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. Oct 24 09:01:52.488400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.490018 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-fbf62387-8874-4564-ae36-aceff6b3c1a3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Returning 404 to user: Instance d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:52.490018 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-fbf62387-8874-4564-ae36-aceff6b3c1a3 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf" status: 404 len: 111 microversion: 2.1 time: 0.043776 Oct 24 09:01:52.491213 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 501/989] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:52 2022] GET /compute/v2.1/servers/d9301ca5-bdaa-4f5d-8bcd-772b8923b1cf => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:01:52.497815 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-dad86672-d3d3-49c7-86b2-dcddbc322683 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '.version_select at 0x7fdfedd96820>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:52.523739 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-dad86672-d3d3-49c7-86b2-dcddbc322683 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-492103469" status: 204 len: 0 microversion: 2.20 time: 0.031041 Oct 24 09:01:52.524658 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 502/990] 10.222.0.39 () {60 vars in 1259 bytes} [Mon Oct 24 09:01:52 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-492103469 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 24 09:01:52.542835 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e306fc7b-c364-4302-b08b-99f5910d88b9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:52.543617 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e306fc7b-c364-4302-b08b-99f5910d88b9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:52.553764 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e306fc7b-c364-4302-b08b-99f5910d88b9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:52.556718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e306fc7b-c364-4302-b08b-99f5910d88b9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:52.558664 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e306fc7b-c364-4302-b08b-99f5910d88b9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:52.559504 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e306fc7b-c364-4302-b08b-99f5910d88b9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:52.560221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e306fc7b-c364-4302-b08b-99f5910d88b9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:52.560860 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e306fc7b-c364-4302-b08b-99f5910d88b9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:52.583496 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-e306fc7b-c364-4302-b08b-99f5910d88b9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] HTTP exception thrown: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e306fc7b-c364-4302-b08b-99f5910d88b9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Request method failure captured: Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: DELETE /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 HTTP/1.1 Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlH_P2cgqdGxTyt_DjhRKpesrNF98h-yweC7dRlZggOYZXXonk3ROJmezNObJqlbezl2V7Pqsl8fDKOVcqEGBVvIZd3l8-ZBbHliak_eivuyadmAWxfis-R_etsiC3KVVo1LvGlKxFu0GFkQBrTFC5gIfs64iGB92PfDe2tNtbCL85U Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:01:52.585667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:01:52.587253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b34f10ec378a4622a8f0b0546e84da7f", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}] Oct 24 09:01:52.587253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:52.587253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:52.587253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:52.587253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:52.587253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:01:52.587253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:01:52.587253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: ebdef1d69bb14837b9a0c28cbba332fc Oct 24 09:01:52.587253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:52.587253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 09:01:52.587253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. Oct 24 09:01:52.587253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': '8f61c54c-9a6d-445b-8a49-056657208b85'} Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:52.588358 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. Oct 24 09:01:52.590802 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.593140 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e306fc7b-c364-4302-b08b-99f5910d88b9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Returning 404 to user: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:52.594622 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e306fc7b-c364-4302-b08b-99f5910d88b9 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "DELETE /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 404 len: 111 microversion: 2.1 time: 0.068916 Oct 24 09:01:52.595815 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 503/991] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 09:01:52 2022] DELETE /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:01:52.605012 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-fc4e2c60-eda2-405c-9de0-40f8cd81e332 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:52.606000 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-fc4e2c60-eda2-405c-9de0-40f8cd81e332 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: 8f61c54c-9a6d-445b-8a49-056657208b85] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:52.612952 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fc4e2c60-eda2-405c-9de0-40f8cd81e332 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:52.613451 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fc4e2c60-eda2-405c-9de0-40f8cd81e332 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:52.613895 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fc4e2c60-eda2-405c-9de0-40f8cd81e332 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:52.614927 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fc4e2c60-eda2-405c-9de0-40f8cd81e332 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:52.615714 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fc4e2c60-eda2-405c-9de0-40f8cd81e332 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:52.616912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fc4e2c60-eda2-405c-9de0-40f8cd81e332 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:52.637866 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-fc4e2c60-eda2-405c-9de0-40f8cd81e332 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] HTTP exception thrown: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-fc4e2c60-eda2-405c-9de0-40f8cd81e332 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Request method failure captured: Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 HTTP/1.1 Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlH_P2cgqdGxTyt_DjhRKpesrNF98h-yweC7dRlZggOYZXXonk3ROJmezNObJqlbezl2V7Pqsl8fDKOVcqEGBVvIZd3l8-ZBbHliak_eivuyadmAWxfis-R_etsiC3KVVo1LvGlKxFu0GFkQBrTFC5gIfs64iGB92PfDe2tNtbCL85U Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:01:52.640151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:01:52.641771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b34f10ec378a4622a8f0b0546e84da7f", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}] Oct 24 09:01:52.641771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:52.641771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:52.641771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:52.641771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:52.641771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:01:52.641771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:01:52.641771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: ebdef1d69bb14837b9a0c28cbba332fc Oct 24 09:01:52.641771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:52.641771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 09:01:52.641771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. Oct 24 09:01:52.641771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': '8f61c54c-9a6d-445b-8a49-056657208b85'} Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:52.642912 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. Oct 24 09:01:52.644343 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:52.647282 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-fc4e2c60-eda2-405c-9de0-40f8cd81e332 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Returning 404 to user: Instance 8f61c54c-9a6d-445b-8a49-056657208b85 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:52.648848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-fc4e2c60-eda2-405c-9de0-40f8cd81e332 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85" status: 404 len: 111 microversion: 2.1 time: 0.048844 Oct 24 09:01:52.650302 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 504/992] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:52 2022] GET /compute/v2.1/servers/8f61c54c-9a6d-445b-8a49-056657208b85 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:01:52.944398 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Flavor limits 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 09:01:52.945351 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Image limits 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 09:01:52.945860 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Flavor pref 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 09:01:52.946289 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Image pref 0:0:0 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 09:01:52.946681 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.virt.hardware [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 09:01:52.956271 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:52.956768 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:52.957204 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:52.957939 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:52.958582 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:52.959014 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:52.977725 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Getting quotas for project 7e9c4dec37674f73912f0d9272ac2446. Resources: {'ram', 'instances', 'cores'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 09:01:52.983280 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.quota [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Getting quotas for user 4f104bf97764426cb67b14bbf39e16be and project 7e9c4dec37674f73912f0d9272ac2446. Resources: {'ram', 'instances', 'cores'} {{(pid=130396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 09:01:52.996876 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Going to run 1 instances... {{(pid=130396) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 09:01:52.999441 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b45a827e-4d67-4f58-abbe-bd39c4cf9586',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 09:01:53.049214 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c279a86f-0866-4454-a684-2b93690d0672 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.858736 Oct 24 09:01:53.049857 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 489/993] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 09:01:52 2022] POST /compute/v2.1/servers => generated 380 bytes in 860 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 09:01:53.061447 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-cbd1fbed-4a12-4a92-af8d-b8e9f6034e68 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:53.062222 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-cbd1fbed-4a12-4a92-af8d-b8e9f6034e68 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:53.137761 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cbd1fbed-4a12-4a92-af8d-b8e9f6034e68 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:53.147043 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-cbd1fbed-4a12-4a92-af8d-b8e9f6034e68 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1258 microversion: 2.1 time: 0.089890 Oct 24 09:01:53.148937 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 505/994] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:53 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1258 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:01:53.860245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-561cf6b5-5577-453c-8acc-01f5fdc99e72 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:53.860861 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-561cf6b5-5577-453c-8acc-01f5fdc99e72 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:53.867955 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-561cf6b5-5577-453c-8acc-01f5fdc99e72 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:53.868633 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-561cf6b5-5577-453c-8acc-01f5fdc99e72 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:53.869077 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-561cf6b5-5577-453c-8acc-01f5fdc99e72 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:53.869850 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-561cf6b5-5577-453c-8acc-01f5fdc99e72 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:53.870973 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-561cf6b5-5577-453c-8acc-01f5fdc99e72 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:53.871467 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-561cf6b5-5577-453c-8acc-01f5fdc99e72 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:53.888535 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-561cf6b5-5577-453c-8acc-01f5fdc99e72 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] HTTP exception thrown: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-561cf6b5-5577-453c-8acc-01f5fdc99e72 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Request method failure captured: Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: DELETE /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 HTTP/1.1 Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlH_P2cgqdGxTyt_DjhRKpesrNF98h-yweC7dRlZggOYZXXonk3ROJmezNObJqlbezl2V7Pqsl8fDKOVcqEGBVvIZd3l8-ZBbHliak_eivuyadmAWxfis-R_etsiC3KVVo1LvGlKxFu0GFkQBrTFC5gIfs64iGB92PfDe2tNtbCL85U Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 09:01:53.890256 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 09:01:53.891916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b34f10ec378a4622a8f0b0546e84da7f", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}] Oct 24 09:01:53.891916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:53.891916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:53.891916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:53.891916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:53.891916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 09:01:53.891916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 09:01:53.891916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: ebdef1d69bb14837b9a0c28cbba332fc Oct 24 09:01:53.891916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:53.891916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 09:01:53.891916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. Oct 24 09:01:53.891916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': 'a85913b0-03e9-4510-9fc7-1514f34c95c1'} Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:53.892855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. Oct 24 09:01:53.897215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:53.898892 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-561cf6b5-5577-453c-8acc-01f5fdc99e72 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Returning 404 to user: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:53.898892 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-561cf6b5-5577-453c-8acc-01f5fdc99e72 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "DELETE /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 404 len: 111 microversion: 2.1 time: 0.038113 Oct 24 09:01:53.898892 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 490/995] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 09:01:53 2022] DELETE /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:01:53.902256 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-717af836-9eaf-445c-837f-fcd67d9e8531 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:53.903105 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-717af836-9eaf-445c-837f-fcd67d9e8531 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] [instance: a85913b0-03e9-4510-9fc7-1514f34c95c1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:53.909869 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-717af836-9eaf-445c-837f-fcd67d9e8531 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:53.910484 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-717af836-9eaf-445c-837f-fcd67d9e8531 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:53.911012 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-717af836-9eaf-445c-837f-fcd67d9e8531 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:53.912601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-717af836-9eaf-445c-837f-fcd67d9e8531 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:53.912994 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-717af836-9eaf-445c-837f-fcd67d9e8531 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:53.913571 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-717af836-9eaf-445c-837f-fcd67d9e8531 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:53.929806 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-717af836-9eaf-445c-837f-fcd67d9e8531 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] HTTP exception thrown: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-717af836-9eaf-445c-837f-fcd67d9e8531 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Request method failure captured: Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 HTTP/1.1 Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlH_P2cgqdGxTyt_DjhRKpesrNF98h-yweC7dRlZggOYZXXonk3ROJmezNObJqlbezl2V7Pqsl8fDKOVcqEGBVvIZd3l8-ZBbHliak_eivuyadmAWxfis-R_etsiC3KVVo1LvGlKxFu0GFkQBrTFC5gIfs64iGB92PfDe2tNtbCL85U Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:01:53.931471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:01:53.933924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_b34f10ec378a4622a8f0b0546e84da7f", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/b34f10ec378a4622a8f0b0546e84da7f"}]}] Oct 24 09:01:53.933924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:53.933924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: b34f10ec378a4622a8f0b0546e84da7f Oct 24 09:01:53.933924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-97597731 Oct 24 09:01:53.933924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:53.933924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:01:53.933924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:01:53.933924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: ebdef1d69bb14837b9a0c28cbba332fc Oct 24 09:01:53.933924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-AttachVolumeNegativeTest-97597731-project Oct 24 09:01:53.933924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 09:01:53.933924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. Oct 24 09:01:53.933924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': 'a85913b0-03e9-4510-9fc7-1514f34c95c1'} Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:53.934812 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. Oct 24 09:01:53.936092 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:53.938668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-717af836-9eaf-445c-837f-fcd67d9e8531 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] Returning 404 to user: Instance a85913b0-03e9-4510-9fc7-1514f34c95c1 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:53.938668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-717af836-9eaf-445c-837f-fcd67d9e8531 tempest-AttachVolumeNegativeTest-97597731 tempest-AttachVolumeNegativeTest-97597731-project] 10.222.0.39 "GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1" status: 404 len: 111 microversion: 2.1 time: 0.036402 Oct 24 09:01:53.938668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 506/996] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:53 2022] GET /compute/v2.1/servers/a85913b0-03e9-4510-9fc7-1514f34c95c1 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:01:54.171044 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-86e0b78b-3d94-43ee-9f5d-c24366d41509 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:54.171667 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-86e0b78b-3d94-43ee-9f5d-c24366d41509 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:54.180351 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-86e0b78b-3d94-43ee-9f5d-c24366d41509 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:54.181051 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-86e0b78b-3d94-43ee-9f5d-c24366d41509 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:54.181592 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-86e0b78b-3d94-43ee-9f5d-c24366d41509 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:54.183271 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-86e0b78b-3d94-43ee-9f5d-c24366d41509 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:54.184032 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-86e0b78b-3d94-43ee-9f5d-c24366d41509 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:54.184719 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-86e0b78b-3d94-43ee-9f5d-c24366d41509 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:54.287640 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-86e0b78b-3d94-43ee-9f5d-c24366d41509 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:54.299074 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-86e0b78b-3d94-43ee-9f5d-c24366d41509 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1310 microversion: 2.1 time: 0.132115 Oct 24 09:01:54.304177 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 491/997] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:54 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1310 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:01:54.607572 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8871d457-733f-4bfd-840a-b5d2f7447a02 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:54.608376 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8871d457-733f-4bfd-840a-b5d2f7447a02 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:54.621619 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8871d457-733f-4bfd-840a-b5d2f7447a02 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:54.623572 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8871d457-733f-4bfd-840a-b5d2f7447a02 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:54.625226 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8871d457-733f-4bfd-840a-b5d2f7447a02 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:54.627299 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8871d457-733f-4bfd-840a-b5d2f7447a02 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:54.627864 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8871d457-733f-4bfd-840a-b5d2f7447a02 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:54.628579 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8871d457-733f-4bfd-840a-b5d2f7447a02 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:54.656611 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-8871d457-733f-4bfd-840a-b5d2f7447a02 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] HTTP exception thrown: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8871d457-733f-4bfd-840a-b5d2f7447a02 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Request method failure captured: Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: DELETE /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 HTTP/1.1 Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlH7mBC-BQLg0H8yK7G2g-HOPUEsdjQL5a9CNStTzz2vRIXjmn3BKvY3yRoT_VlpqjhmiOscl52jQCwEMfyH6tF1HUFCQtCOEAYK8cH5PiF_zTNaUGNGpBjSQoGhunfKNz-_3ckGEvv273V4IyJwd9U0Vvic8VBlr37q6YrJuFn9p-Y Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Openstack-Nova-Api-Version: 2.20 Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: 7ff48b26c576495f8bb8c88d20d246cc Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:01:54.658221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:01:54.659735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/7ff48b26c576495f8bb8c88d20d246cc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_7ff48b26c576495f8bb8c88d20d246cc", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/7ff48b26c576495f8bb8c88d20d246cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/7ff48b26c576495f8bb8c88d20d246cc"}]}] Oct 24 09:01:54.659735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 09:01:54.659735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: 7ff48b26c576495f8bb8c88d20d246cc Oct 24 09:01:54.659735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 09:01:54.659735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-AttachVolumeShelveTestJSON-160255499-project Oct 24 09:01:54.659735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:01:54.659735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:01:54.659735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: 552c23b8456a40ec8e01297aca7965d3 Oct 24 09:01:54.659735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-AttachVolumeShelveTestJSON-160255499-project Oct 24 09:01:54.659735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 09:01:54.659735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. Oct 24 09:01:54.659735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': 'e72acc7a-decb-46c7-9c51-f731e2c413b3'} Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:54.660564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. Oct 24 09:01:54.662452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:54.664903 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8871d457-733f-4bfd-840a-b5d2f7447a02 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Returning 404 to user: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:54.664903 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8871d457-733f-4bfd-840a-b5d2f7447a02 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "DELETE /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 404 len: 111 microversion: 2.20 time: 0.056894 Oct 24 09:01:54.664903 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 507/998] 10.222.0.39 () {60 vars in 1280 bytes} [Mon Oct 24 09:01:54 2022] DELETE /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 111 bytes in 62 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 24 09:01:54.671326 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6ec68d1e-ed57-4740-b25a-e59004f89447 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:54.671964 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6ec68d1e-ed57-4740-b25a-e59004f89447 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: e72acc7a-decb-46c7-9c51-f731e2c413b3] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:54.679061 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6ec68d1e-ed57-4740-b25a-e59004f89447 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:54.681044 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6ec68d1e-ed57-4740-b25a-e59004f89447 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:54.681512 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6ec68d1e-ed57-4740-b25a-e59004f89447 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:54.682438 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6ec68d1e-ed57-4740-b25a-e59004f89447 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:54.683631 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6ec68d1e-ed57-4740-b25a-e59004f89447 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:54.684267 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6ec68d1e-ed57-4740-b25a-e59004f89447 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:54.701715 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-6ec68d1e-ed57-4740-b25a-e59004f89447 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] HTTP exception thrown: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6ec68d1e-ed57-4740-b25a-e59004f89447 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Request method failure captured: Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 HTTP/1.1 Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlH7mBC-BQLg0H8yK7G2g-HOPUEsdjQL5a9CNStTzz2vRIXjmn3BKvY3yRoT_VlpqjhmiOscl52jQCwEMfyH6tF1HUFCQtCOEAYK8cH5PiF_zTNaUGNGpBjSQoGhunfKNz-_3ckGEvv273V4IyJwd9U0Vvic8VBlr37q6YrJuFn9p-Y Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Openstack-Nova-Api-Version: 2.20 Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 7ff48b26c576495f8bb8c88d20d246cc Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 09:01:54.704505 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 09:01:54.706245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/7ff48b26c576495f8bb8c88d20d246cc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_7ff48b26c576495f8bb8c88d20d246cc", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/7ff48b26c576495f8bb8c88d20d246cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/7ff48b26c576495f8bb8c88d20d246cc"}]}] Oct 24 09:01:54.706245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 09:01:54.706245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 7ff48b26c576495f8bb8c88d20d246cc Oct 24 09:01:54.706245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 09:01:54.706245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-AttachVolumeShelveTestJSON-160255499-project Oct 24 09:01:54.706245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 09:01:54.706245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 09:01:54.706245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 552c23b8456a40ec8e01297aca7965d3 Oct 24 09:01:54.706245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-AttachVolumeShelveTestJSON-160255499-project Oct 24 09:01:54.706245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 09:01:54.706245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. Oct 24 09:01:54.706245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': 'e72acc7a-decb-46c7-9c51-f731e2c413b3'} Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:54.707206 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. Oct 24 09:01:54.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:54.710714 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6ec68d1e-ed57-4740-b25a-e59004f89447 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Returning 404 to user: Instance e72acc7a-decb-46c7-9c51-f731e2c413b3 could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:54.712272 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6ec68d1e-ed57-4740-b25a-e59004f89447 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3" status: 404 len: 111 microversion: 2.20 time: 0.044830 Oct 24 09:01:54.713447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 492/999] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:01:54 2022] GET /compute/v2.1/servers/e72acc7a-decb-46c7-9c51-f731e2c413b3 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 24 09:01:55.325523 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c6f5eabd-3b6e-4519-b701-a55b7d6a9363 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:55.327098 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c6f5eabd-3b6e-4519-b701-a55b7d6a9363 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:55.343964 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c6f5eabd-3b6e-4519-b701-a55b7d6a9363 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:55.347264 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c6f5eabd-3b6e-4519-b701-a55b7d6a9363 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:55.348036 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c6f5eabd-3b6e-4519-b701-a55b7d6a9363 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:55.360758 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c6f5eabd-3b6e-4519-b701-a55b7d6a9363 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:55.361623 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c6f5eabd-3b6e-4519-b701-a55b7d6a9363 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:55.362363 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c6f5eabd-3b6e-4519-b701-a55b7d6a9363 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:55.545949 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c6f5eabd-3b6e-4519-b701-a55b7d6a9363 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:55.569438 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c6f5eabd-3b6e-4519-b701-a55b7d6a9363 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1316 microversion: 2.1 time: 0.248674 Oct 24 09:01:55.580604 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 508/1000] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:55 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1316 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:01:55.903342 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-3617279a-72bb-442a-8e74-94b4879ef332 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:55.906024 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-3617279a-72bb-442a-8e74-94b4879ef332 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:55.946560 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3617279a-72bb-442a-8e74-94b4879ef332 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:55.948710 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3617279a-72bb-442a-8e74-94b4879ef332 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:55.950995 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3617279a-72bb-442a-8e74-94b4879ef332 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:55.952925 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3617279a-72bb-442a-8e74-94b4879ef332 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:55.957717 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3617279a-72bb-442a-8e74-94b4879ef332 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:55.959122 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3617279a-72bb-442a-8e74-94b4879ef332 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:56.026610 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-3617279a-72bb-442a-8e74-94b4879ef332 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] HTTP exception thrown: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-3617279a-72bb-442a-8e74-94b4879ef332 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Request method failure captured: Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: DELETE /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 HTTP/1.1 Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlH7mBC-BQLg0H8yK7G2g-HOPUEsdjQL5a9CNStTzz2vRIXjmn3BKvY3yRoT_VlpqjhmiOscl52jQCwEMfyH6tF1HUFCQtCOEAYK8cH5PiF_zTNaUGNGpBjSQoGhunfKNz-_3ckGEvv273V4IyJwd9U0Vvic8VBlr37q6YrJuFn9p-Y Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Openstack-Nova-Api-Version: 2.20 Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 7ff48b26c576495f8bb8c88d20d246cc Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 09:01:56.032968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 09:01:56.034962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/7ff48b26c576495f8bb8c88d20d246cc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_7ff48b26c576495f8bb8c88d20d246cc", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/7ff48b26c576495f8bb8c88d20d246cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/7ff48b26c576495f8bb8c88d20d246cc"}]}] Oct 24 09:01:56.034962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 09:01:56.034962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 7ff48b26c576495f8bb8c88d20d246cc Oct 24 09:01:56.034962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 09:01:56.034962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-AttachVolumeShelveTestJSON-160255499-project Oct 24 09:01:56.034962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 09:01:56.034962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 09:01:56.034962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 552c23b8456a40ec8e01297aca7965d3 Oct 24 09:01:56.034962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-AttachVolumeShelveTestJSON-160255499-project Oct 24 09:01:56.034962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 09:01:56.034962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. Oct 24 09:01:56.034962 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': '71e106ba-ad54-4e9c-8e42-9477c8db8cf6'} Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:56.035957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. Oct 24 09:01:56.037603 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:01:56.043063 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-3617279a-72bb-442a-8e74-94b4879ef332 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Returning 404 to user: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:56.045626 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-3617279a-72bb-442a-8e74-94b4879ef332 tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "DELETE /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 404 len: 111 microversion: 2.20 time: 0.152766 Oct 24 09:01:56.048553 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 493/1001] 10.222.0.39 () {60 vars in 1280 bytes} [Mon Oct 24 09:01:55 2022] DELETE /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 111 bytes in 156 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 24 09:01:56.058527 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c90177d8-a149-48bb-a7f2-0a7959e0095b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:56.059513 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c90177d8-a149-48bb-a7f2-0a7959e0095b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] [instance: 71e106ba-ad54-4e9c-8e42-9477c8db8cf6] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:56.082908 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c90177d8-a149-48bb-a7f2-0a7959e0095b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:56.096951 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c90177d8-a149-48bb-a7f2-0a7959e0095b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:56.097534 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c90177d8-a149-48bb-a7f2-0a7959e0095b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:56.098488 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c90177d8-a149-48bb-a7f2-0a7959e0095b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:56.099126 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c90177d8-a149-48bb-a7f2-0a7959e0095b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:56.102333 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c90177d8-a149-48bb-a7f2-0a7959e0095b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:56.165575 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-c90177d8-a149-48bb-a7f2-0a7959e0095b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] HTTP exception thrown: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c90177d8-a149-48bb-a7f2-0a7959e0095b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Request method failure captured: Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 HTTP/1.1 Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlH7mBC-BQLg0H8yK7G2g-HOPUEsdjQL5a9CNStTzz2vRIXjmn3BKvY3yRoT_VlpqjhmiOscl52jQCwEMfyH6tF1HUFCQtCOEAYK8cH5PiF_zTNaUGNGpBjSQoGhunfKNz-_3ckGEvv273V4IyJwd9U0Vvic8VBlr37q6YrJuFn9p-Y Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Openstack-Nova-Api-Version: 2.20 Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: 7ff48b26c576495f8bb8c88d20d246cc Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:01:56.167307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:01:56.169220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/7ff48b26c576495f8bb8c88d20d246cc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_7ff48b26c576495f8bb8c88d20d246cc", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/7ff48b26c576495f8bb8c88d20d246cc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/7ff48b26c576495f8bb8c88d20d246cc"}]}] Oct 24 09:01:56.169220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 09:01:56.169220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: 7ff48b26c576495f8bb8c88d20d246cc Oct 24 09:01:56.169220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-160255499 Oct 24 09:01:56.169220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-AttachVolumeShelveTestJSON-160255499-project Oct 24 09:01:56.169220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:01:56.169220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:01:56.169220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: 552c23b8456a40ec8e01297aca7965d3 Oct 24 09:01:56.169220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-AttachVolumeShelveTestJSON-160255499-project Oct 24 09:01:56.169220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 09:01:56.169220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. Oct 24 09:01:56.169220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': '71e106ba-ad54-4e9c-8e42-9477c8db8cf6'} Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:01:56.170103 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. Oct 24 09:01:56.171554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:01:56.172973 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c90177d8-a149-48bb-a7f2-0a7959e0095b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] Returning 404 to user: Instance 71e106ba-ad54-4e9c-8e42-9477c8db8cf6 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:01:56.172973 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c90177d8-a149-48bb-a7f2-0a7959e0095b tempest-AttachVolumeShelveTestJSON-160255499 tempest-AttachVolumeShelveTestJSON-160255499-project] 10.222.0.39 "GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6" status: 404 len: 111 microversion: 2.20 time: 0.115795 Oct 24 09:01:56.172973 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 509/1002] 10.222.0.39 () {60 vars in 1277 bytes} [Mon Oct 24 09:01:56 2022] GET /compute/v2.1/servers/71e106ba-ad54-4e9c-8e42-9477c8db8cf6 => generated 111 bytes in 117 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 24 09:01:56.604231 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-25ae27f1-7308-4c7c-9d50-1bb41f9deeb3 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:56.606935 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-25ae27f1-7308-4c7c-9d50-1bb41f9deeb3 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:56.627009 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-25ae27f1-7308-4c7c-9d50-1bb41f9deeb3 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:56.644232 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-25ae27f1-7308-4c7c-9d50-1bb41f9deeb3 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:56.644988 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-25ae27f1-7308-4c7c-9d50-1bb41f9deeb3 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:56.647421 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-25ae27f1-7308-4c7c-9d50-1bb41f9deeb3 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:56.652749 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-25ae27f1-7308-4c7c-9d50-1bb41f9deeb3 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:56.653484 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-25ae27f1-7308-4c7c-9d50-1bb41f9deeb3 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:57.097146 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-25ae27f1-7308-4c7c-9d50-1bb41f9deeb3 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:57.110656 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-25ae27f1-7308-4c7c-9d50-1bb41f9deeb3 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.516930 Oct 24 09:01:57.133065 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 494/1003] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:56 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 541 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:01:58.139636 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f92a4366-aa4d-4b7b-952f-bb482f30ab6e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:58.146943 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f92a4366-aa4d-4b7b-952f-bb482f30ab6e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:58.159389 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f92a4366-aa4d-4b7b-952f-bb482f30ab6e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:58.162094 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f92a4366-aa4d-4b7b-952f-bb482f30ab6e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:58.166083 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f92a4366-aa4d-4b7b-952f-bb482f30ab6e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:58.168097 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f92a4366-aa4d-4b7b-952f-bb482f30ab6e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:58.170352 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f92a4366-aa4d-4b7b-952f-bb482f30ab6e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:58.170990 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f92a4366-aa4d-4b7b-952f-bb482f30ab6e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:58.691769 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f92a4366-aa4d-4b7b-952f-bb482f30ab6e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:01:58.706732 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f92a4366-aa4d-4b7b-952f-bb482f30ab6e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.572351 Oct 24 09:01:58.708557 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 510/1004] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:58 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:01:59.389169 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-7e6a062d-b6d3-4c56-8c3f-e514daaf8978 req-062ba494-0866-47c6-8506-fe9f73bfb613 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a7260920-be84-4a74-a18e-5b47d53e3db1", "tag": "b8947813-7d62-4e04-8aa8-8ba174d01def"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:01:59.414404 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-7e6a062d-b6d3-4c56-8c3f-e514daaf8978 req-062ba494-0866-47c6-8506-fe9f73bfb613 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:59.417462 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-7e6a062d-b6d3-4c56-8c3f-e514daaf8978 req-062ba494-0866-47c6-8506-fe9f73bfb613 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:59.418840 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-7e6a062d-b6d3-4c56-8c3f-e514daaf8978 req-062ba494-0866-47c6-8506-fe9f73bfb613 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:59.449451 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-7e6a062d-b6d3-4c56-8c3f-e514daaf8978 req-062ba494-0866-47c6-8506-fe9f73bfb613 service nova] Creating event network-changed:b8947813-7d62-4e04-8aa8-8ba174d01def for instance a7260920-be84-4a74-a18e-5b47d53e3db1 on hv-cinder-maste Oct 24 09:01:59.459325 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-7e6a062d-b6d3-4c56-8c3f-e514daaf8978 req-062ba494-0866-47c6-8506-fe9f73bfb613 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.075248 Oct 24 09:01:59.463842 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 495/1005] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:01:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:01:59.734867 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f116cbbb-c2ac-48c7-ab2c-3a1aac67eb44 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:01:59.736035 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f116cbbb-c2ac-48c7-ab2c-3a1aac67eb44 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:01:59.747734 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f116cbbb-c2ac-48c7-ab2c-3a1aac67eb44 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:59.750789 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f116cbbb-c2ac-48c7-ab2c-3a1aac67eb44 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:59.751428 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f116cbbb-c2ac-48c7-ab2c-3a1aac67eb44 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:01:59.753166 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f116cbbb-c2ac-48c7-ab2c-3a1aac67eb44 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:01:59.755100 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f116cbbb-c2ac-48c7-ab2c-3a1aac67eb44 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:01:59.756829 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f116cbbb-c2ac-48c7-ab2c-3a1aac67eb44 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:00.354108 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f116cbbb-c2ac-48c7-ab2c-3a1aac67eb44 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:00.366675 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f116cbbb-c2ac-48c7-ab2c-3a1aac67eb44 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.636914 Oct 24 09:02:00.374880 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 511/1006] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:01:59 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 646 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:01.404582 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2d73a1ae-e50c-40a9-8689-53750a0a9f74 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:01.405616 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2d73a1ae-e50c-40a9-8689-53750a0a9f74 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:01.438038 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2d73a1ae-e50c-40a9-8689-53750a0a9f74 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:01.440862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2d73a1ae-e50c-40a9-8689-53750a0a9f74 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:01.445620 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2d73a1ae-e50c-40a9-8689-53750a0a9f74 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:01.447342 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2d73a1ae-e50c-40a9-8689-53750a0a9f74 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:01.453900 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2d73a1ae-e50c-40a9-8689-53750a0a9f74 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:01.454482 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2d73a1ae-e50c-40a9-8689-53750a0a9f74 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:01.976742 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2d73a1ae-e50c-40a9-8689-53750a0a9f74 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:01.988558 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2d73a1ae-e50c-40a9-8689-53750a0a9f74 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.589210 Oct 24 09:02:01.992014 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 496/1007] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:01 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 09:02:03.017071 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-4f163271-f35c-46e3-839d-83ee2e786ed0 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:03.019522 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-4f163271-f35c-46e3-839d-83ee2e786ed0 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:03.033326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4f163271-f35c-46e3-839d-83ee2e786ed0 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:03.034143 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4f163271-f35c-46e3-839d-83ee2e786ed0 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:03.045544 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4f163271-f35c-46e3-839d-83ee2e786ed0 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:03.047798 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4f163271-f35c-46e3-839d-83ee2e786ed0 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:03.049509 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4f163271-f35c-46e3-839d-83ee2e786ed0 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:03.049975 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-4f163271-f35c-46e3-839d-83ee2e786ed0 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:03.142518 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=6 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:02:03.395284 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-4f163271-f35c-46e3-839d-83ee2e786ed0 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:03.461201 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-4f163271-f35c-46e3-839d-83ee2e786ed0 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.446120 Oct 24 09:02:03.463546 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 512/1008] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:03 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:04.492884 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-14be4f56-9d74-4a2e-8fc0-408dcd74f98e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:04.493651 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-14be4f56-9d74-4a2e-8fc0-408dcd74f98e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:04.497609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=2 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:02:04.500688 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=6 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:02:04.523049 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-14be4f56-9d74-4a2e-8fc0-408dcd74f98e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:04.523812 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-14be4f56-9d74-4a2e-8fc0-408dcd74f98e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:04.525796 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-14be4f56-9d74-4a2e-8fc0-408dcd74f98e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:04.529212 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-14be4f56-9d74-4a2e-8fc0-408dcd74f98e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:04.531010 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-14be4f56-9d74-4a2e-8fc0-408dcd74f98e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:04.532758 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-14be4f56-9d74-4a2e-8fc0-408dcd74f98e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:05.143784 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-14be4f56-9d74-4a2e-8fc0-408dcd74f98e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:05.153816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-14be4f56-9d74-4a2e-8fc0-408dcd74f98e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.665884 Oct 24 09:02:05.156701 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 497/1009] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:04 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 670 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:05.187170 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-83f7e0ee-9a0b-4c15-af02-0a670d6d4a69 tempest-VolumesGetTestJSON-2091904113 tempest-VolumesGetTestJSON-2091904113-project] Action: 'create', calling method: .version_select at 0x7fdfedc0e790>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-815765221", "metadata": {"Type": "work"}}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:02:05.895241 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-83f7e0ee-9a0b-4c15-af02-0a670d6d4a69 tempest-VolumesGetTestJSON-2091904113 tempest-VolumesGetTestJSON-2091904113-project] 10.222.0.39 "POST /compute/v2.1/os-volumes" status: 200 len: 352 microversion: 2.1 time: 0.952192 Oct 24 09:02:05.896780 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 513/1010] 10.222.0.39 () {60 vars in 1156 bytes} [Mon Oct 24 09:02:04 2022] POST /compute/v2.1/os-volumes => generated 352 bytes in 955 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 24 09:02:05.911352 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8371f466-5ab5-40eb-bc87-d3a5a0bad3b4 tempest-VolumesGetTestJSON-2091904113 tempest-VolumesGetTestJSON-2091904113-project] Calling method '.version_select at 0x7fdfedd5fb80>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:06.078346 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8371f466-5ab5-40eb-bc87-d3a5a0bad3b4 tempest-VolumesGetTestJSON-2091904113 tempest-VolumesGetTestJSON-2091904113-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9" status: 200 len: 352 microversion: 2.1 time: 0.171452 Oct 24 09:02:06.079242 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 498/1011] 10.222.0.39 () {58 vars in 1245 bytes} [Mon Oct 24 09:02:05 2022] GET /compute/v2.1/os-volumes/44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9 => generated 352 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:02:06.186962 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8ec61664-e4cf-4da6-a09b-b994d4cbe796 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:06.187660 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8ec61664-e4cf-4da6-a09b-b994d4cbe796 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:06.218732 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ec61664-e4cf-4da6-a09b-b994d4cbe796 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:06.220903 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ec61664-e4cf-4da6-a09b-b994d4cbe796 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:06.222797 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ec61664-e4cf-4da6-a09b-b994d4cbe796 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:06.222797 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ec61664-e4cf-4da6-a09b-b994d4cbe796 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:06.223026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ec61664-e4cf-4da6-a09b-b994d4cbe796 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:06.223437 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8ec61664-e4cf-4da6-a09b-b994d4cbe796 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:06.519829 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8ec61664-e4cf-4da6-a09b-b994d4cbe796 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:06.559063 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8ec61664-e4cf-4da6-a09b-b994d4cbe796 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.377699 Oct 24 09:02:06.576326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 514/1012] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:06 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:07.093663 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-43a8a0fc-3539-4774-84f3-7db941e20436 tempest-VolumesGetTestJSON-2091904113 tempest-VolumesGetTestJSON-2091904113-project] Calling method '.version_select at 0x7fdff16660d0>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:07.145497 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-43a8a0fc-3539-4774-84f3-7db941e20436 tempest-VolumesGetTestJSON-2091904113 tempest-VolumesGetTestJSON-2091904113-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9" status: 200 len: 353 microversion: 2.1 time: 0.056211 Oct 24 09:02:07.146220 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 499/1013] 10.222.0.39 () {58 vars in 1245 bytes} [Mon Oct 24 09:02:07 2022] GET /compute/v2.1/os-volumes/44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9 => generated 353 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:02:07.162498 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e32c9056-e8c0-40d3-8550-6e2061b4956f tempest-VolumesGetTestJSON-2091904113 tempest-VolumesGetTestJSON-2091904113-project] Calling method '.version_select at 0x7fdfedd20dc0>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:07.219797 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e32c9056-e8c0-40d3-8550-6e2061b4956f tempest-VolumesGetTestJSON-2091904113 tempest-VolumesGetTestJSON-2091904113-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9" status: 200 len: 353 microversion: 2.1 time: 0.063739 Oct 24 09:02:07.220502 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 515/1014] 10.222.0.39 () {58 vars in 1245 bytes} [Mon Oct 24 09:02:07 2022] GET /compute/v2.1/os-volumes/44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9 => generated 353 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:02:07.240695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b6c6a2e5-bc00-403f-b7f9-1e5f60c4bc76 tempest-VolumesGetTestJSON-2091904113 tempest-VolumesGetTestJSON-2091904113-project] Calling method '.version_select at 0x7fdfedef2ca0>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:07.324974 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b6c6a2e5-bc00-403f-b7f9-1e5f60c4bc76 tempest-VolumesGetTestJSON-2091904113 tempest-VolumesGetTestJSON-2091904113-project] 10.222.0.39 "DELETE /compute/v2.1/os-volumes/44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9" status: 202 len: 0 microversion: 2.1 time: 0.089143 Oct 24 09:02:07.325805 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 500/1015] 10.222.0.39 () {58 vars in 1248 bytes} [Mon Oct 24 09:02:07 2022] DELETE /compute/v2.1/os-volumes/44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9 => generated 0 bytes in 91 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 24 09:02:07.335038 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ead63af3-a7dc-490d-9ebb-3be190bf7649 tempest-VolumesGetTestJSON-2091904113 tempest-VolumesGetTestJSON-2091904113-project] Calling method '.version_select at 0x7fdfedfdb280>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:07.403710 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ead63af3-a7dc-490d-9ebb-3be190bf7649 tempest-VolumesGetTestJSON-2091904113 tempest-VolumesGetTestJSON-2091904113-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9" status: 200 len: 352 microversion: 2.1 time: 0.073673 Oct 24 09:02:07.404458 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 516/1016] 10.222.0.39 () {58 vars in 1245 bytes} [Mon Oct 24 09:02:07 2022] GET /compute/v2.1/os-volumes/44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9 => generated 352 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 09:02:07.583984 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0fc78316-b37c-4825-bf5a-5e8b1af69f3d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:07.584761 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-0fc78316-b37c-4825-bf5a-5e8b1af69f3d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:07.593802 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fc78316-b37c-4825-bf5a-5e8b1af69f3d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:07.594310 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fc78316-b37c-4825-bf5a-5e8b1af69f3d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:07.595071 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fc78316-b37c-4825-bf5a-5e8b1af69f3d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:07.595856 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fc78316-b37c-4825-bf5a-5e8b1af69f3d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:07.596628 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fc78316-b37c-4825-bf5a-5e8b1af69f3d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:07.597111 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0fc78316-b37c-4825-bf5a-5e8b1af69f3d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:07.775955 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0fc78316-b37c-4825-bf5a-5e8b1af69f3d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:07.785957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-0fc78316-b37c-4825-bf5a-5e8b1af69f3d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.206321 Oct 24 09:02:07.793051 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 501/1017] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:07 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:07.940486 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8df1b334-1199-425d-b95e-f15c399d0c70 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Action: 'create', calling method: .version_select at 0x7fdfedc9e940>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-996350897"}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:02:08.259390 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8df1b334-1199-425d-b95e-f15c399d0c70 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 0.440829 Oct 24 09:02:08.260024 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 517/1018] 10.222.0.39 () {60 vars in 1155 bytes} [Mon Oct 24 09:02:07 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 442 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 24 09:02:08.274588 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-aaf3852a-9f22-4ae5-857e-a0e5d02ac358 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdff1666820>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:08.327142 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-aaf3852a-9f22-4ae5-857e-a0e5d02ac358 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e" status: 200 len: 344 microversion: 2.1 time: 0.056298 Oct 24 09:02:08.327142 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 502/1019] 10.222.0.39 () {58 vars in 1245 bytes} [Mon Oct 24 09:02:08 2022] GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e => generated 344 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 09:02:08.418139 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b4911c18-aff1-4a1d-8fc6-7cbbb2be41e9 tempest-VolumesGetTestJSON-2091904113 tempest-VolumesGetTestJSON-2091904113-project] Calling method '.version_select at 0x7fdfedc223a0>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:08.445205 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-b4911c18-aff1-4a1d-8fc6-7cbbb2be41e9 tempest-VolumesGetTestJSON-2091904113 tempest-VolumesGetTestJSON-2091904113-project] HTTP exception thrown: Volume 44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9 could not be found. Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b4911c18-aff1-4a1d-8fc6-7cbbb2be41e9 tempest-VolumesGetTestJSON-2091904113 tempest-VolumesGetTestJSON-2091904113-project] Request method failure captured: Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/os-volumes/44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9 HTTP/1.1 Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlSM099Qt63YvB0uEkHZ-OFqO5PD7nPZu3vitknKe6dm0CTp24Mqm1PZLby8nsVvLohl2V7JV69r0bIIHNsSoHSMKzC-YgJ2qH3hn5al_ScnyFRGvs607NG_zvvRinTprCl7-wDi42qLOFCrx-fsVZTAf-ILm70B38IrYjsHvdNv-XU Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: 6762a2261e54498e82855f614d98562f Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-VolumesGetTestJSON-2091904113 Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:02:08.447159 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:02:08.449848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/6762a2261e54498e82855f614d98562f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_6762a2261e54498e82855f614d98562f", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/6762a2261e54498e82855f614d98562f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/6762a2261e54498e82855f614d98562f"}]}] Oct 24 09:02:08.449848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-VolumesGetTestJSON-2091904113 Oct 24 09:02:08.449848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: 6762a2261e54498e82855f614d98562f Oct 24 09:02:08.449848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-VolumesGetTestJSON-2091904113 Oct 24 09:02:08.449848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-VolumesGetTestJSON-2091904113-project Oct 24 09:02:08.449848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:02:08.449848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:02:08.449848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: 661867b9dc1541d18d5224cc292ab010 Oct 24 09:02:08.449848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-VolumesGetTestJSON-2091904113-project Oct 24 09:02:08.449848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: Oct 24 09:02:08.449848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Volume 44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9 could not be found. Oct 24 09:02:08.449848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': '44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9'} Oct 24 09:02:08.449848 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9 could not be found. (HTTP 404) (Request-ID: req-8e8213d8-0113-4092-8275-34111e6355d6) Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:02:08.451079 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 24 09:02:08.452718 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9 could not be found. Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 24 09:02:08.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:02:08.455460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9 could not be found. Oct 24 09:02:08.455460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:08.455460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:02:08.455460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:08.455460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:08.455460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:02:08.455460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:02:08.455460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:02:08.455460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:02:08.455460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9 could not be found. Oct 24 09:02:08.455460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:08.455460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b4911c18-aff1-4a1d-8fc6-7cbbb2be41e9 tempest-VolumesGetTestJSON-2091904113 tempest-VolumesGetTestJSON-2091904113-project] Returning 404 to user: Volume 44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:02:08.455460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b4911c18-aff1-4a1d-8fc6-7cbbb2be41e9 tempest-VolumesGetTestJSON-2091904113 tempest-VolumesGetTestJSON-2091904113-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9" status: 404 len: 109 microversion: 2.1 time: 0.034644 Oct 24 09:02:08.455460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 518/1020] 10.222.0.39 () {58 vars in 1245 bytes} [Mon Oct 24 09:02:08 2022] GET /compute/v2.1/os-volumes/44bd1bd3-2e12-41b0-a80f-7cd20a99b8b9 => generated 109 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:02:08.808743 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-62009c3e-e3f9-4f8c-addf-f6c197fee464 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:08.809353 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-62009c3e-e3f9-4f8c-addf-f6c197fee464 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:08.815622 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62009c3e-e3f9-4f8c-addf-f6c197fee464 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:08.816121 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62009c3e-e3f9-4f8c-addf-f6c197fee464 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:08.816589 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62009c3e-e3f9-4f8c-addf-f6c197fee464 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:08.817300 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62009c3e-e3f9-4f8c-addf-f6c197fee464 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:08.817787 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62009c3e-e3f9-4f8c-addf-f6c197fee464 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:08.818245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-62009c3e-e3f9-4f8c-addf-f6c197fee464 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:09.038101 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-62009c3e-e3f9-4f8c-addf-f6c197fee464 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:09.050167 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-62009c3e-e3f9-4f8c-addf-f6c197fee464 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.245203 Oct 24 09:02:09.051245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 503/1021] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:08 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:09.343530 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-715bedee-feef-44e4-b301-a65df259cc46 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfedc60940>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:09.408583 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-715bedee-feef-44e4-b301-a65df259cc46 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e" status: 200 len: 345 microversion: 2.1 time: 0.071894 Oct 24 09:02:09.409153 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 519/1022] 10.222.0.39 () {58 vars in 1245 bytes} [Mon Oct 24 09:02:09 2022] GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e => generated 345 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:02:09.428545 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b0d1f011-4779-42e2-a140-9c315cff9098 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Action: 'create', calling method: .version_select at 0x7fdff1666550>, body: {"snapshot": {"volume_id": "d26f17cf-758e-4ee9-a9e6-9ff1246fc38e", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1910827120"}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:02:09.623223 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b0d1f011-4779-42e2-a140-9c315cff9098 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.204782 Oct 24 09:02:09.623811 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 504/1023] 10.222.0.39 () {60 vars in 1162 bytes} [Mon Oct 24 09:02:09 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 09:02:09.635755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-68c19c7a-99db-41b5-b18f-0b5e2bb863cd tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfedc22f70>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:09.675256 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-68c19c7a-99db-41b5-b18f-0b5e2bb863cd tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d" status: 200 len: 287 microversion: 2.1 time: 0.043554 Oct 24 09:02:09.676024 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 520/1024] 10.222.0.39 () {58 vars in 1251 bytes} [Mon Oct 24 09:02:09 2022] GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d => generated 287 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:02:09.688174 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-14436035-6780-4e76-8364-36614f246b9d tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfedeefca0>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:09.723528 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-14436035-6780-4e76-8364-36614f246b9d tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.039113 Oct 24 09:02:09.723528 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 505/1025] 10.222.0.39 () {58 vars in 1140 bytes} [Mon Oct 24 09:02:09 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:02:09.735753 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f724dc47-927e-4102-b87a-fa5aebee5cc4 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfedc600d0>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:09.768445 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f724dc47-927e-4102-b87a-fa5aebee5cc4 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d" status: 200 len: 287 microversion: 2.1 time: 0.037467 Oct 24 09:02:09.769025 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 521/1026] 10.222.0.39 () {58 vars in 1251 bytes} [Mon Oct 24 09:02:09 2022] GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d => generated 287 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:02:10.074495 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ca24d65b-9f8e-44dc-8907-c087c87703a7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:10.075418 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ca24d65b-9f8e-44dc-8907-c087c87703a7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:10.082435 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ca24d65b-9f8e-44dc-8907-c087c87703a7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:10.086046 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ca24d65b-9f8e-44dc-8907-c087c87703a7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:10.086046 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ca24d65b-9f8e-44dc-8907-c087c87703a7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:10.086046 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ca24d65b-9f8e-44dc-8907-c087c87703a7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:10.086046 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ca24d65b-9f8e-44dc-8907-c087c87703a7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:10.086375 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ca24d65b-9f8e-44dc-8907-c087c87703a7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:10.226233 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ca24d65b-9f8e-44dc-8907-c087c87703a7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:10.234965 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ca24d65b-9f8e-44dc-8907-c087c87703a7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.165531 Oct 24 09:02:10.235452 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 506/1027] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:10 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:10.783024 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ef286723-98d5-41ea-8ff0-4e8ecf90dda2 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfedd97dc0>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:10.847256 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ef286723-98d5-41ea-8ff0-4e8ecf90dda2 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d" status: 200 len: 287 microversion: 2.1 time: 0.070161 Oct 24 09:02:10.847874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 522/1028] 10.222.0.39 () {58 vars in 1251 bytes} [Mon Oct 24 09:02:10 2022] GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d => generated 287 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:02:11.258544 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8cda9607-aafa-4f35-bd52-23e7909216fd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:11.259093 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8cda9607-aafa-4f35-bd52-23e7909216fd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:11.266713 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8cda9607-aafa-4f35-bd52-23e7909216fd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:11.267189 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8cda9607-aafa-4f35-bd52-23e7909216fd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:11.267620 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8cda9607-aafa-4f35-bd52-23e7909216fd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:11.268357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8cda9607-aafa-4f35-bd52-23e7909216fd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:11.268806 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8cda9607-aafa-4f35-bd52-23e7909216fd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:11.269223 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8cda9607-aafa-4f35-bd52-23e7909216fd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:11.408732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8cda9607-aafa-4f35-bd52-23e7909216fd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:11.416564 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8cda9607-aafa-4f35-bd52-23e7909216fd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.161924 Oct 24 09:02:11.417149 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 507/1029] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:11 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:11.861826 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-efa2aff4-f1fb-42a9-bc13-634e9814ebcd tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfede0e1f0>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:11.892914 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-efa2aff4-f1fb-42a9-bc13-634e9814ebcd tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d" status: 200 len: 287 microversion: 2.1 time: 0.035735 Oct 24 09:02:11.893479 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 523/1030] 10.222.0.39 () {58 vars in 1251 bytes} [Mon Oct 24 09:02:11 2022] GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d => generated 287 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:02:12.439693 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-27c65723-949e-469a-bc06-12223a42e2ad tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:12.440337 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-27c65723-949e-469a-bc06-12223a42e2ad tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:12.446804 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-27c65723-949e-469a-bc06-12223a42e2ad tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:12.447290 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-27c65723-949e-469a-bc06-12223a42e2ad tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:12.447726 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-27c65723-949e-469a-bc06-12223a42e2ad tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:12.448462 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-27c65723-949e-469a-bc06-12223a42e2ad tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:12.449686 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-27c65723-949e-469a-bc06-12223a42e2ad tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:12.450121 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-27c65723-949e-469a-bc06-12223a42e2ad tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:12.578713 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-27c65723-949e-469a-bc06-12223a42e2ad tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:12.586380 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-27c65723-949e-469a-bc06-12223a42e2ad tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.150570 Oct 24 09:02:12.586937 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 508/1031] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:12 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:12.907389 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-76a28149-5bfb-44b3-959a-4f6a88fe2ab7 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfedc60dc0>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:12.937650 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-76a28149-5bfb-44b3-959a-4f6a88fe2ab7 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d" status: 200 len: 287 microversion: 2.1 time: 0.035691 Oct 24 09:02:12.938215 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 524/1032] 10.222.0.39 () {58 vars in 1251 bytes} [Mon Oct 24 09:02:12 2022] GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d => generated 287 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:02:13.609544 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-d2fc2a6f-4c2e-4555-bb70-5160feb37871 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:13.610151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-d2fc2a6f-4c2e-4555-bb70-5160feb37871 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:13.616852 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d2fc2a6f-4c2e-4555-bb70-5160feb37871 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:13.617333 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d2fc2a6f-4c2e-4555-bb70-5160feb37871 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:13.617766 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d2fc2a6f-4c2e-4555-bb70-5160feb37871 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:13.618460 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d2fc2a6f-4c2e-4555-bb70-5160feb37871 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:13.618910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d2fc2a6f-4c2e-4555-bb70-5160feb37871 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:13.619331 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d2fc2a6f-4c2e-4555-bb70-5160feb37871 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:13.622105 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:02:13.759078 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d2fc2a6f-4c2e-4555-bb70-5160feb37871 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:13.767649 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-d2fc2a6f-4c2e-4555-bb70-5160feb37871 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.162313 Oct 24 09:02:13.768440 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 509/1033] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:13 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:13.951949 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a41fdec0-0d5a-47bb-98fb-c030360fa5f0 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfedc22310>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:13.983251 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a41fdec0-0d5a-47bb-98fb-c030360fa5f0 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d" status: 200 len: 287 microversion: 2.1 time: 0.036663 Oct 24 09:02:13.983833 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 525/1034] 10.222.0.39 () {58 vars in 1251 bytes} [Mon Oct 24 09:02:13 2022] GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d => generated 287 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:02:14.790792 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-49e9de75-ab73-4a47-a68d-e90b50f5eb88 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:14.791338 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-49e9de75-ab73-4a47-a68d-e90b50f5eb88 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:14.799281 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-49e9de75-ab73-4a47-a68d-e90b50f5eb88 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:14.799766 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-49e9de75-ab73-4a47-a68d-e90b50f5eb88 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:14.800246 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-49e9de75-ab73-4a47-a68d-e90b50f5eb88 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:14.800948 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-49e9de75-ab73-4a47-a68d-e90b50f5eb88 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:14.801403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-49e9de75-ab73-4a47-a68d-e90b50f5eb88 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:14.801831 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-49e9de75-ab73-4a47-a68d-e90b50f5eb88 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:14.934721 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-49e9de75-ab73-4a47-a68d-e90b50f5eb88 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:14.942406 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-49e9de75-ab73-4a47-a68d-e90b50f5eb88 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.155635 Oct 24 09:02:14.942967 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 510/1035] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:14 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:14.997254 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-77580fd4-6346-49bf-9cee-e301e73b959b tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfede0e280>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:15.028761 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-77580fd4-6346-49bf-9cee-e301e73b959b tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d" status: 200 len: 288 microversion: 2.1 time: 0.036017 Oct 24 09:02:15.029341 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 526/1036] 10.222.0.39 () {58 vars in 1251 bytes} [Mon Oct 24 09:02:14 2022] GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d => generated 288 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:02:15.040441 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-1032f009-6abf-4e0b-a1f3-6a26b182ea6a tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfee024ca0>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:15.103482 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-1032f009-6abf-4e0b-a1f3-6a26b182ea6a tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "DELETE /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d" status: 202 len: 0 microversion: 2.1 time: 0.066576 Oct 24 09:02:15.104673 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 511/1037] 10.222.0.39 () {58 vars in 1254 bytes} [Mon Oct 24 09:02:15 2022] DELETE /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d => generated 0 bytes in 68 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 09:02:15.114061 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-31ab12ca-a7b6-4d4b-a70e-49cfc1516657 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfedc225e0>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:15.148992 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-31ab12ca-a7b6-4d4b-a70e-49cfc1516657 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d" status: 200 len: 287 microversion: 2.1 time: 0.039116 Oct 24 09:02:15.149706 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 527/1038] 10.222.0.39 () {58 vars in 1251 bytes} [Mon Oct 24 09:02:15 2022] GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d => generated 287 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:02:15.969512 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-bbf55ddc-921d-4f8b-8665-9dfe59c190a6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:15.970646 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-bbf55ddc-921d-4f8b-8665-9dfe59c190a6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:15.982957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bbf55ddc-921d-4f8b-8665-9dfe59c190a6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:15.983861 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bbf55ddc-921d-4f8b-8665-9dfe59c190a6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:15.984810 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bbf55ddc-921d-4f8b-8665-9dfe59c190a6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:15.985553 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bbf55ddc-921d-4f8b-8665-9dfe59c190a6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:15.986033 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bbf55ddc-921d-4f8b-8665-9dfe59c190a6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:15.986758 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-bbf55ddc-921d-4f8b-8665-9dfe59c190a6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:16.164125 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-781c1f47-eb07-4b37-a668-8662db2f2703 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfedc600d0>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:16.190238 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-bbf55ddc-921d-4f8b-8665-9dfe59c190a6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:16.195224 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=61,nova_api:UPDATE=1,nova_api:DELETE=2,nova_api:INSERT=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:02:16.225451 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-781c1f47-eb07-4b37-a668-8662db2f2703 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] HTTP exception thrown: Snapshot 465f896d-6162-44ca-afcc-f2da74369d6d could not be found. Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-781c1f47-eb07-4b37-a668-8662db2f2703 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Request method failure captured: Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d HTTP/1.1 Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlSP57qrd9uCUMRBbZ4vNsiKrmpflgz0RsqQyxRy10uO81JNt64xtT_HvilXKWIivfvzqSJtNMmkKHOZ6R2roMuk639xTg_87bBpYMq_V43s3Rm-0e2XqiXmZTmmhDEqHBThXPePut0aKEI1fkPrPCmk-a11ugHTduK93KYeIvcirPM Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: 51f1284fd166460bbd7243217b6da7d1 Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1109557823 Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:02:16.229178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:02:16.231006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/51f1284fd166460bbd7243217b6da7d1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_51f1284fd166460bbd7243217b6da7d1", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/51f1284fd166460bbd7243217b6da7d1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/51f1284fd166460bbd7243217b6da7d1"}]}] Oct 24 09:02:16.231006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1109557823 Oct 24 09:02:16.231006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: 51f1284fd166460bbd7243217b6da7d1 Oct 24 09:02:16.231006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1109557823 Oct 24 09:02:16.231006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-VolumesSnapshotsTestJSON-1109557823-project Oct 24 09:02:16.231006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:02:16.231006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:02:16.231006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: 5e063f7f37ae43cdb2016f3bab247184 Oct 24 09:02:16.231006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1109557823-project Oct 24 09:02:16.231006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: Oct 24 09:02:16.231006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Snapshot 465f896d-6162-44ca-afcc-f2da74369d6d could not be found. Oct 24 09:02:16.231006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': '465f896d-6162-44ca-afcc-f2da74369d6d'} Oct 24 09:02:16.231006 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 465f896d-6162-44ca-afcc-f2da74369d6d could not be found. (HTTP 404) (Request-ID: req-60743c11-26d5-4211-977f-c5873893b888) Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:16.231896 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 24 09:02:16.235603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 465f896d-6162-44ca-afcc-f2da74369d6d could not be found. Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Oct 24 09:02:16.237708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:02:16.239288 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 465f896d-6162-44ca-afcc-f2da74369d6d could not be found. Oct 24 09:02:16.239288 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:16.239288 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:02:16.239288 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:16.239288 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:16.239288 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:02:16.239288 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:02:16.239288 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:02:16.239288 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:02:16.239288 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 465f896d-6162-44ca-afcc-f2da74369d6d could not be found. Oct 24 09:02:16.239288 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:16.243082 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-781c1f47-eb07-4b37-a668-8662db2f2703 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Returning 404 to user: Snapshot 465f896d-6162-44ca-afcc-f2da74369d6d could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:02:16.243238 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-bbf55ddc-921d-4f8b-8665-9dfe59c190a6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.281416 Oct 24 09:02:16.243869 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 512/1039] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:15 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 09:02:16.245181 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-781c1f47-eb07-4b37-a668-8662db2f2703 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d" status: 404 len: 111 microversion: 2.1 time: 0.085489 Oct 24 09:02:16.247056 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 528/1040] 10.222.0.39 () {58 vars in 1251 bytes} [Mon Oct 24 09:02:16 2022] GET /compute/v2.1/os-snapshots/465f896d-6162-44ca-afcc-f2da74369d6d => generated 111 bytes in 88 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 24 09:02:16.260445 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f3ea59d4-0adb-4eeb-ac72-cb29df4a8954 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfedc4f670>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:16.373352 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f3ea59d4-0adb-4eeb-ac72-cb29df4a8954 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "DELETE /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e" status: 202 len: 0 microversion: 2.1 time: 0.119703 Oct 24 09:02:16.377835 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 513/1041] 10.222.0.39 () {58 vars in 1248 bytes} [Mon Oct 24 09:02:16 2022] DELETE /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e => generated 0 bytes in 122 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 09:02:16.385102 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f923a193-e4b1-4530-8f4b-fd05c153999a tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfede0e5e0>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:16.442307 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f923a193-e4b1-4530-8f4b-fd05c153999a tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e" status: 200 len: 344 microversion: 2.1 time: 0.062183 Oct 24 09:02:16.442995 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 529/1042] 10.222.0.39 () {58 vars in 1245 bytes} [Mon Oct 24 09:02:16 2022] GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e => generated 344 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:02:17.271419 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-71d26cd4-9bdf-424f-b420-b00fc913bb27 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:17.271979 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-71d26cd4-9bdf-424f-b420-b00fc913bb27 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:17.279030 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-71d26cd4-9bdf-424f-b420-b00fc913bb27 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:17.279522 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-71d26cd4-9bdf-424f-b420-b00fc913bb27 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:17.279957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-71d26cd4-9bdf-424f-b420-b00fc913bb27 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:17.280685 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-71d26cd4-9bdf-424f-b420-b00fc913bb27 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:17.281131 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-71d26cd4-9bdf-424f-b420-b00fc913bb27 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:17.281545 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-71d26cd4-9bdf-424f-b420-b00fc913bb27 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:17.455589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=80,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:02:17.460224 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b6fd061e-1b93-4fb1-bada-b8ccdbb37f8f tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfedd97700>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:17.481175 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-71d26cd4-9bdf-424f-b420-b00fc913bb27 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:17.489991 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-71d26cd4-9bdf-424f-b420-b00fc913bb27 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.223115 Oct 24 09:02:17.490562 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 514/1043] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:17 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:17.514189 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b6fd061e-1b93-4fb1-bada-b8ccdbb37f8f tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e" status: 200 len: 344 microversion: 2.1 time: 0.059189 Oct 24 09:02:17.519892 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 530/1044] 10.222.0.39 () {58 vars in 1245 bytes} [Mon Oct 24 09:02:17 2022] GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e => generated 344 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 09:02:18.514661 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-fd2ff3ac-25c4-4ca4-9d3c-3e7b7b2cfbe8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:18.515312 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-fd2ff3ac-25c4-4ca4-9d3c-3e7b7b2cfbe8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:18.517620 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=57,nova_api:INSERT=3 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:02:18.524300 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fd2ff3ac-25c4-4ca4-9d3c-3e7b7b2cfbe8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:18.524942 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fd2ff3ac-25c4-4ca4-9d3c-3e7b7b2cfbe8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:18.525392 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fd2ff3ac-25c4-4ca4-9d3c-3e7b7b2cfbe8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:18.526112 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fd2ff3ac-25c4-4ca4-9d3c-3e7b7b2cfbe8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:18.526577 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fd2ff3ac-25c4-4ca4-9d3c-3e7b7b2cfbe8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:18.527531 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fd2ff3ac-25c4-4ca4-9d3c-3e7b7b2cfbe8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:18.530834 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-2b2590e4-8657-41fe-9888-4616e4bb8ada tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfedd97040>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:18.603964 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-2b2590e4-8657-41fe-9888-4616e4bb8ada tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e" status: 200 len: 344 microversion: 2.1 time: 0.078562 Oct 24 09:02:18.604744 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 531/1045] 10.222.0.39 () {58 vars in 1245 bytes} [Mon Oct 24 09:02:18 2022] GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e => generated 344 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 09:02:18.683185 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fd2ff3ac-25c4-4ca4-9d3c-3e7b7b2cfbe8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:18.691522 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-fd2ff3ac-25c4-4ca4-9d3c-3e7b7b2cfbe8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.180730 Oct 24 09:02:18.692102 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 515/1046] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:18 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:19.620347 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-615fbb6c-0c7d-46d4-a42f-060ec0580e45 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfede5c040>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:19.673008 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-615fbb6c-0c7d-46d4-a42f-060ec0580e45 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e" status: 200 len: 344 microversion: 2.1 time: 0.057261 Oct 24 09:02:19.673829 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 532/1047] 10.222.0.39 () {58 vars in 1245 bytes} [Mon Oct 24 09:02:19 2022] GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e => generated 344 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 09:02:19.731490 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-190fe026-2b23-498a-bd3c-43475c0fe2d5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:19.732234 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-190fe026-2b23-498a-bd3c-43475c0fe2d5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:19.739790 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-190fe026-2b23-498a-bd3c-43475c0fe2d5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:19.740312 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-190fe026-2b23-498a-bd3c-43475c0fe2d5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:19.740759 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-190fe026-2b23-498a-bd3c-43475c0fe2d5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:19.741749 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-190fe026-2b23-498a-bd3c-43475c0fe2d5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:19.742204 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-190fe026-2b23-498a-bd3c-43475c0fe2d5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:19.742627 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-190fe026-2b23-498a-bd3c-43475c0fe2d5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:19.881752 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-190fe026-2b23-498a-bd3c-43475c0fe2d5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:19.890379 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-190fe026-2b23-498a-bd3c-43475c0fe2d5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.163120 Oct 24 09:02:19.890966 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 516/1048] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:19 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:20.688476 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f1ff8341-bb08-4f27-ac73-0c80a53d0d74 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfedc22f70>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:20.716612 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-f1ff8341-bb08-4f27-ac73-0c80a53d0d74 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] HTTP exception thrown: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f1ff8341-bb08-4f27-ac73-0c80a53d0d74 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Request method failure captured: Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e HTTP/1.1 Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlSP57qrd9uCUMRBbZ4vNsiKrmpflgz0RsqQyxRy10uO81JNt64xtT_HvilXKWIivfvzqSJtNMmkKHOZ6R2roMuk639xTg_87bBpYMq_V43s3Rm-0e2XqiXmZTmmhDEqHBThXPePut0aKEI1fkPrPCmk-a11ugHTduK93KYeIvcirPM Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: 51f1284fd166460bbd7243217b6da7d1 Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1109557823 Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:02:20.718699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:02:20.720379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/51f1284fd166460bbd7243217b6da7d1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_51f1284fd166460bbd7243217b6da7d1", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/51f1284fd166460bbd7243217b6da7d1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/51f1284fd166460bbd7243217b6da7d1"}]}] Oct 24 09:02:20.720379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1109557823 Oct 24 09:02:20.720379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: 51f1284fd166460bbd7243217b6da7d1 Oct 24 09:02:20.720379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1109557823 Oct 24 09:02:20.720379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-VolumesSnapshotsTestJSON-1109557823-project Oct 24 09:02:20.720379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:02:20.720379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:02:20.720379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: 5e063f7f37ae43cdb2016f3bab247184 Oct 24 09:02:20.720379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1109557823-project Oct 24 09:02:20.720379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: Oct 24 09:02:20.720379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. Oct 24 09:02:20.720379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': 'd26f17cf-758e-4ee9-a9e6-9ff1246fc38e'} Oct 24 09:02:20.720379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. (HTTP 404) (Request-ID: req-dfaa516b-e0d2-429e-80bb-e3ec2c8774e4) Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:02:20.721391 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 24 09:02:20.722804 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 24 09:02:20.724515 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:02:20.726026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. Oct 24 09:02:20.726026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.726026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:02:20.726026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.726026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:20.726026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:02:20.726026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:02:20.726026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:02:20.726026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:02:20.726026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. Oct 24 09:02:20.726026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.726026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f1ff8341-bb08-4f27-ac73-0c80a53d0d74 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Returning 404 to user: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:02:20.726026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f1ff8341-bb08-4f27-ac73-0c80a53d0d74 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e" status: 404 len: 109 microversion: 2.1 time: 0.037165 Oct 24 09:02:20.726026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 533/1049] 10.222.0.39 () {58 vars in 1245 bytes} [Mon Oct 24 09:02:20 2022] GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e => generated 109 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:02:20.734567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0d631ae6-d4ed-4010-a723-01b9851ecb52 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfedeb2820>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:20.764901 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-0d631ae6-d4ed-4010-a723-01b9851ecb52 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] HTTP exception thrown: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0d631ae6-d4ed-4010-a723-01b9851ecb52 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Request method failure captured: Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: DELETE /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e HTTP/1.1 Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlSP57qrd9uCUMRBbZ4vNsiKrmpflgz0RsqQyxRy10uO81JNt64xtT_HvilXKWIivfvzqSJtNMmkKHOZ6R2roMuk639xTg_87bBpYMq_V43s3Rm-0e2XqiXmZTmmhDEqHBThXPePut0aKEI1fkPrPCmk-a11ugHTduK93KYeIvcirPM Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 51f1284fd166460bbd7243217b6da7d1 Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1109557823 Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 09:02:20.767199 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 09:02:20.768910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/51f1284fd166460bbd7243217b6da7d1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_51f1284fd166460bbd7243217b6da7d1", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/51f1284fd166460bbd7243217b6da7d1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/51f1284fd166460bbd7243217b6da7d1"}]}] Oct 24 09:02:20.768910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1109557823 Oct 24 09:02:20.768910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 51f1284fd166460bbd7243217b6da7d1 Oct 24 09:02:20.768910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1109557823 Oct 24 09:02:20.768910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-VolumesSnapshotsTestJSON-1109557823-project Oct 24 09:02:20.768910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 09:02:20.768910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 09:02:20.768910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 5e063f7f37ae43cdb2016f3bab247184 Oct 24 09:02:20.768910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1109557823-project Oct 24 09:02:20.768910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: Oct 24 09:02:20.768910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. Oct 24 09:02:20.768910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': 'd26f17cf-758e-4ee9-a9e6-9ff1246fc38e'} Oct 24 09:02:20.768910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. (HTTP 404) (Request-ID: req-e79c3292-0f9b-403e-91b9-eabf5475838f) Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:02:20.769947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 24 09:02:20.771412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 24 09:02:20.772977 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:02:20.774636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. Oct 24 09:02:20.774636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.774636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:02:20.774636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.774636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:20.774636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:02:20.774636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:02:20.774636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:02:20.774636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:02:20.774636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. Oct 24 09:02:20.774636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.774636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0d631ae6-d4ed-4010-a723-01b9851ecb52 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Returning 404 to user: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:02:20.774636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-0d631ae6-d4ed-4010-a723-01b9851ecb52 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "DELETE /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e" status: 404 len: 109 microversion: 2.1 time: 0.038291 Oct 24 09:02:20.775727 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 517/1050] 10.222.0.39 () {58 vars in 1248 bytes} [Mon Oct 24 09:02:20 2022] DELETE /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e => generated 109 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:02:20.777272 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a569eae3-c065-4691-a8af-2fa06eb90475 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Calling method '.version_select at 0x7fdfede96550>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:20.806151 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-a569eae3-c065-4691-a8af-2fa06eb90475 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] HTTP exception thrown: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a569eae3-c065-4691-a8af-2fa06eb90475 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Request method failure captured: Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e HTTP/1.1 Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlSP57qrd9uCUMRBbZ4vNsiKrmpflgz0RsqQyxRy10uO81JNt64xtT_HvilXKWIivfvzqSJtNMmkKHOZ6R2roMuk639xTg_87bBpYMq_V43s3Rm-0e2XqiXmZTmmhDEqHBThXPePut0aKEI1fkPrPCmk-a11ugHTduK93KYeIvcirPM Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: 51f1284fd166460bbd7243217b6da7d1 Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1109557823 Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:02:20.808689 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:02:20.810266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/51f1284fd166460bbd7243217b6da7d1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_51f1284fd166460bbd7243217b6da7d1", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/51f1284fd166460bbd7243217b6da7d1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/51f1284fd166460bbd7243217b6da7d1"}]}] Oct 24 09:02:20.810266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1109557823 Oct 24 09:02:20.810266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: 51f1284fd166460bbd7243217b6da7d1 Oct 24 09:02:20.810266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1109557823 Oct 24 09:02:20.810266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-VolumesSnapshotsTestJSON-1109557823-project Oct 24 09:02:20.810266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:02:20.810266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:02:20.810266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: 5e063f7f37ae43cdb2016f3bab247184 Oct 24 09:02:20.810266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1109557823-project Oct 24 09:02:20.810266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: Oct 24 09:02:20.810266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. Oct 24 09:02:20.810266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': 'd26f17cf-758e-4ee9-a9e6-9ff1246fc38e'} Oct 24 09:02:20.810266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. (HTTP 404) (Request-ID: req-5013e345-8435-4118-8048-232cab298592) Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:02:20.811267 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 24 09:02:20.812767 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 24 09:02:20.814169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:02:20.815759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. Oct 24 09:02:20.815759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.815759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:02:20.815759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.815759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:02:20.815759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:02:20.815759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:02:20.815759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:02:20.815759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:02:20.815759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. Oct 24 09:02:20.815759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:02:20.815759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a569eae3-c065-4691-a8af-2fa06eb90475 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] Returning 404 to user: Volume d26f17cf-758e-4ee9-a9e6-9ff1246fc38e could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:02:20.815759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a569eae3-c065-4691-a8af-2fa06eb90475 tempest-VolumesSnapshotsTestJSON-1109557823 tempest-VolumesSnapshotsTestJSON-1109557823-project] 10.222.0.39 "GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e" status: 404 len: 109 microversion: 2.1 time: 0.036848 Oct 24 09:02:20.815759 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 534/1051] 10.222.0.39 () {58 vars in 1245 bytes} [Mon Oct 24 09:02:20 2022] GET /compute/v2.1/os-volumes/d26f17cf-758e-4ee9-a9e6-9ff1246fc38e => generated 109 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:02:20.915848 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-eff323b6-0ac3-4122-bcaf-2506709089f4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:20.916499 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-eff323b6-0ac3-4122-bcaf-2506709089f4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:20.925311 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-eff323b6-0ac3-4122-bcaf-2506709089f4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:20.925807 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-eff323b6-0ac3-4122-bcaf-2506709089f4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:20.926251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-eff323b6-0ac3-4122-bcaf-2506709089f4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:20.927182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-eff323b6-0ac3-4122-bcaf-2506709089f4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:20.927637 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-eff323b6-0ac3-4122-bcaf-2506709089f4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:20.928269 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-eff323b6-0ac3-4122-bcaf-2506709089f4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:21.080492 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-eff323b6-0ac3-4122-bcaf-2506709089f4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:21.086574 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-eff323b6-0ac3-4122-bcaf-2506709089f4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.174501 Oct 24 09:02:21.087660 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 518/1052] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:20 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:21.623049 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-6e1ff648-e8ca-4836-b81c-cffa83c29c29 req-20dd00c7-6031-46b9-a132-660da9b7eec0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a7260920-be84-4a74-a18e-5b47d53e3db1", "name": "network-vif-plugged", "status": "completed", "tag": "b8947813-7d62-4e04-8aa8-8ba174d01def"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:02:21.633977 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-6e1ff648-e8ca-4836-b81c-cffa83c29c29 req-20dd00c7-6031-46b9-a132-660da9b7eec0 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:21.634210 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-6e1ff648-e8ca-4836-b81c-cffa83c29c29 req-20dd00c7-6031-46b9-a132-660da9b7eec0 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:21.634680 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-6e1ff648-e8ca-4836-b81c-cffa83c29c29 req-20dd00c7-6031-46b9-a132-660da9b7eec0 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:21.646471 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-6e1ff648-e8ca-4836-b81c-cffa83c29c29 req-20dd00c7-6031-46b9-a132-660da9b7eec0 service nova] Creating event network-vif-plugged:b8947813-7d62-4e04-8aa8-8ba174d01def for instance a7260920-be84-4a74-a18e-5b47d53e3db1 on hv-cinder-maste Oct 24 09:02:21.652034 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-6e1ff648-e8ca-4836-b81c-cffa83c29c29 req-20dd00c7-6031-46b9-a132-660da9b7eec0 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.197011 Oct 24 09:02:21.652702 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 535/1053] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:02:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:02:22.111191 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7fb01754-eb47-41bb-b9b6-21075c984756 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:22.112017 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7fb01754-eb47-41bb-b9b6-21075c984756 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:22.121597 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb01754-eb47-41bb-b9b6-21075c984756 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:22.122790 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb01754-eb47-41bb-b9b6-21075c984756 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:22.123913 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb01754-eb47-41bb-b9b6-21075c984756 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:22.125114 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb01754-eb47-41bb-b9b6-21075c984756 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:22.125621 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb01754-eb47-41bb-b9b6-21075c984756 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:22.126148 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7fb01754-eb47-41bb-b9b6-21075c984756 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:22.271747 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7fb01754-eb47-41bb-b9b6-21075c984756 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:22.279630 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7fb01754-eb47-41bb-b9b6-21075c984756 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.172614 Oct 24 09:02:22.280225 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 519/1054] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:22 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:23.304524 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-5256b506-4c09-4c65-b4a3-a03469325179 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:23.305244 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-5256b506-4c09-4c65-b4a3-a03469325179 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:23.313618 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5256b506-4c09-4c65-b4a3-a03469325179 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:23.313960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5256b506-4c09-4c65-b4a3-a03469325179 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:23.314408 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5256b506-4c09-4c65-b4a3-a03469325179 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:23.315125 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5256b506-4c09-4c65-b4a3-a03469325179 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:23.315588 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5256b506-4c09-4c65-b4a3-a03469325179 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:23.316055 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5256b506-4c09-4c65-b4a3-a03469325179 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:23.456871 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-5256b506-4c09-4c65-b4a3-a03469325179 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:23.464623 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-5256b506-4c09-4c65-b4a3-a03469325179 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.164927 Oct 24 09:02:23.465626 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 536/1055] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:23 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:24.494628 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-916f7bd9-87ef-48da-a67f-212106d164f1 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:24.497343 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-916f7bd9-87ef-48da-a67f-212106d164f1 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:24.506024 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-916f7bd9-87ef-48da-a67f-212106d164f1 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:24.506798 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-916f7bd9-87ef-48da-a67f-212106d164f1 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:24.508107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-916f7bd9-87ef-48da-a67f-212106d164f1 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:24.509050 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-916f7bd9-87ef-48da-a67f-212106d164f1 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:24.509722 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-916f7bd9-87ef-48da-a67f-212106d164f1 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:24.510541 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-916f7bd9-87ef-48da-a67f-212106d164f1 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:24.705735 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-916f7bd9-87ef-48da-a67f-212106d164f1 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:24.715828 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-916f7bd9-87ef-48da-a67f-212106d164f1 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.229354 Oct 24 09:02:24.717190 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 520/1056] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:24 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:25.741032 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a0f734f2-bbbe-426b-b2f0-27a4b6e5b261 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:25.742405 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a0f734f2-bbbe-426b-b2f0-27a4b6e5b261 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:25.752155 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0f734f2-bbbe-426b-b2f0-27a4b6e5b261 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:25.752906 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0f734f2-bbbe-426b-b2f0-27a4b6e5b261 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:25.753554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0f734f2-bbbe-426b-b2f0-27a4b6e5b261 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:25.754484 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0f734f2-bbbe-426b-b2f0-27a4b6e5b261 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:25.755169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0f734f2-bbbe-426b-b2f0-27a4b6e5b261 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:25.755742 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a0f734f2-bbbe-426b-b2f0-27a4b6e5b261 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:25.893661 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a0f734f2-bbbe-426b-b2f0-27a4b6e5b261 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:25.901276 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a0f734f2-bbbe-426b-b2f0-27a4b6e5b261 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.164349 Oct 24 09:02:25.902508 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 537/1057] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:25 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:26.925153 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f43b072b-866b-4009-9a84-a03c9073f527 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:26.926759 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f43b072b-866b-4009-9a84-a03c9073f527 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:26.934956 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f43b072b-866b-4009-9a84-a03c9073f527 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:26.935447 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f43b072b-866b-4009-9a84-a03c9073f527 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:26.935880 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f43b072b-866b-4009-9a84-a03c9073f527 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:26.936639 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f43b072b-866b-4009-9a84-a03c9073f527 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:26.937086 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f43b072b-866b-4009-9a84-a03c9073f527 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:26.937735 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f43b072b-866b-4009-9a84-a03c9073f527 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:27.087528 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f43b072b-866b-4009-9a84-a03c9073f527 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:27.095284 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f43b072b-866b-4009-9a84-a03c9073f527 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.173863 Oct 24 09:02:27.096287 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 521/1058] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:26 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:28.120430 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c91f2505-5b7e-4c79-9244-c1b0fec3b375 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:28.121797 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c91f2505-5b7e-4c79-9244-c1b0fec3b375 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:28.129098 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c91f2505-5b7e-4c79-9244-c1b0fec3b375 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:28.129622 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c91f2505-5b7e-4c79-9244-c1b0fec3b375 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:28.130371 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c91f2505-5b7e-4c79-9244-c1b0fec3b375 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:28.131054 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c91f2505-5b7e-4c79-9244-c1b0fec3b375 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:28.131554 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c91f2505-5b7e-4c79-9244-c1b0fec3b375 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:28.132014 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c91f2505-5b7e-4c79-9244-c1b0fec3b375 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:28.305527 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c91f2505-5b7e-4c79-9244-c1b0fec3b375 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:28.314909 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c91f2505-5b7e-4c79-9244-c1b0fec3b375 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.199159 Oct 24 09:02:28.315592 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 538/1059] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:28 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:29.339741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-1d34c021-bd4e-472f-a527-65d89cb69162 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:29.341245 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-1d34c021-bd4e-472f-a527-65d89cb69162 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:29.347533 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1d34c021-bd4e-472f-a527-65d89cb69162 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:29.348018 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1d34c021-bd4e-472f-a527-65d89cb69162 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:29.348473 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1d34c021-bd4e-472f-a527-65d89cb69162 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:29.349176 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1d34c021-bd4e-472f-a527-65d89cb69162 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:29.349622 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1d34c021-bd4e-472f-a527-65d89cb69162 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:29.350037 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1d34c021-bd4e-472f-a527-65d89cb69162 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:29.540380 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-1d34c021-bd4e-472f-a527-65d89cb69162 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:29.556398 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-1d34c021-bd4e-472f-a527-65d89cb69162 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.221001 Oct 24 09:02:29.557710 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 522/1060] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:29 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:30.585268 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-26064239-8c10-4bcb-84b6-d53b86684bb8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:30.585822 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-26064239-8c10-4bcb-84b6-d53b86684bb8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:30.593925 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26064239-8c10-4bcb-84b6-d53b86684bb8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:30.594449 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26064239-8c10-4bcb-84b6-d53b86684bb8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:30.594916 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26064239-8c10-4bcb-84b6-d53b86684bb8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:30.595686 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26064239-8c10-4bcb-84b6-d53b86684bb8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:30.596166 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26064239-8c10-4bcb-84b6-d53b86684bb8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:30.596606 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-26064239-8c10-4bcb-84b6-d53b86684bb8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:30.772966 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-26064239-8c10-4bcb-84b6-d53b86684bb8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:30.781440 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-26064239-8c10-4bcb-84b6-d53b86684bb8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1358 microversion: 2.1 time: 0.201608 Oct 24 09:02:30.786272 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 539/1061] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:30 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1358 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:31.815384 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c5ddb9e3-5d87-466e-9e20-67c885a48f7e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:31.816158 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c5ddb9e3-5d87-466e-9e20-67c885a48f7e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:31.823430 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c5ddb9e3-5d87-466e-9e20-67c885a48f7e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:31.824148 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c5ddb9e3-5d87-466e-9e20-67c885a48f7e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:31.825236 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c5ddb9e3-5d87-466e-9e20-67c885a48f7e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:31.826580 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c5ddb9e3-5d87-466e-9e20-67c885a48f7e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:31.827269 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c5ddb9e3-5d87-466e-9e20-67c885a48f7e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:31.828257 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c5ddb9e3-5d87-466e-9e20-67c885a48f7e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:31.977371 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c5ddb9e3-5d87-466e-9e20-67c885a48f7e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:31.987847 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c5ddb9e3-5d87-466e-9e20-67c885a48f7e tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1547 microversion: 2.1 time: 0.176621 Oct 24 09:02:31.996948 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 523/1062] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:31 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1547 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:32.012880 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d625b2a8-4ba7-4080-9423-b67db39f9260 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:32.013509 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d625b2a8-4ba7-4080-9423-b67db39f9260 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:32.022626 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d625b2a8-4ba7-4080-9423-b67db39f9260 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:32.023197 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d625b2a8-4ba7-4080-9423-b67db39f9260 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:32.023719 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d625b2a8-4ba7-4080-9423-b67db39f9260 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:32.024542 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d625b2a8-4ba7-4080-9423-b67db39f9260 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:32.025088 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d625b2a8-4ba7-4080-9423-b67db39f9260 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:32.025594 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d625b2a8-4ba7-4080-9423-b67db39f9260 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:32.179770 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d625b2a8-4ba7-4080-9423-b67db39f9260 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:32.187861 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d625b2a8-4ba7-4080-9423-b67db39f9260 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1547 microversion: 2.1 time: 0.179732 Oct 24 09:02:32.188715 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 540/1063] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:32 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1547 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:32.210813 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e5ddc53f-fd5e-4792-ac5d-6224427af8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:32.216461 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e5ddc53f-fd5e-4792-ac5d-6224427af8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:32.217104 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e5ddc53f-fd5e-4792-ac5d-6224427af8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Searching by: {'deleted': False, 'project_id': '7e9c4dec37674f73912f0d9272ac2446'} {{(pid=130396) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 24 09:02:32.223836 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e5ddc53f-fd5e-4792-ac5d-6224427af8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:32.224347 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e5ddc53f-fd5e-4792-ac5d-6224427af8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:32.224989 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e5ddc53f-fd5e-4792-ac5d-6224427af8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:32.225710 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e5ddc53f-fd5e-4792-ac5d-6224427af8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:32.226155 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e5ddc53f-fd5e-4792-ac5d-6224427af8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:32.226572 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e5ddc53f-fd5e-4792-ac5d-6224427af8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:32.320970 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.multi_cell_list [None req-e5ddc53f-fd5e-4792-ac5d-6224427af8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=130396) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 24 09:02:32.422143 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e5ddc53f-fd5e-4792-ac5d-6224427af8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.215214 Oct 24 09:02:32.422721 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 524/1064] 10.222.0.39 () {58 vars in 1125 bytes} [Mon Oct 24 09:02:32 2022] GET /compute/v2.1/servers => generated 343 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:02:32.435203 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-38e78b5d-e3ab-4c72-900a-08cc6b0dbccf tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:02:32.436091 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-38e78b5d-e3ab-4c72-900a-08cc6b0dbccf tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:32.450971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-38e78b5d-e3ab-4c72-900a-08cc6b0dbccf tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:32.451590 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-38e78b5d-e3ab-4c72-900a-08cc6b0dbccf tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:32.452532 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-38e78b5d-e3ab-4c72-900a-08cc6b0dbccf tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:32.453023 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-38e78b5d-e3ab-4c72-900a-08cc6b0dbccf tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:32.453552 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-38e78b5d-e3ab-4c72-900a-08cc6b0dbccf tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:32.454088 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-38e78b5d-e3ab-4c72-900a-08cc6b0dbccf tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:32.623711 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-38e78b5d-e3ab-4c72-900a-08cc6b0dbccf tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:02:32.639919 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-38e78b5d-e3ab-4c72-900a-08cc6b0dbccf tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1547 microversion: 2.1 time: 0.202783 Oct 24 09:02:32.639919 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 541/1065] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:02:32 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1547 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:02:36.536156 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6d388fb9-97b3-4405-8d9f-5a70b18443df tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "8c130732-ebfb-404c-b2b9-691de5349448"}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:02:36.539307 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6d388fb9-97b3-4405-8d9f-5a70b18443df tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:02:36.553665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d388fb9-97b3-4405-8d9f-5a70b18443df tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:36.554520 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d388fb9-97b3-4405-8d9f-5a70b18443df tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:36.555772 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d388fb9-97b3-4405-8d9f-5a70b18443df tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:36.557786 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d388fb9-97b3-4405-8d9f-5a70b18443df tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:02:36.559758 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d388fb9-97b3-4405-8d9f-5a70b18443df tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:02:36.562295 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6d388fb9-97b3-4405-8d9f-5a70b18443df tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:02:36.919599 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6d388fb9-97b3-4405-8d9f-5a70b18443df tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "POST /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.389440 Oct 24 09:02:36.920498 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 525/1066] 10.222.0.39 () {60 vars in 1323 bytes} [Mon Oct 24 09:02:36 2022] POST /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/os-volume_attachments => generated 194 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:03:00.808185 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1167500606", "imageRef": "bba331ba-faf5-4a56-81d9-6e84e9609665", "flavorRef": "452", "networks": [{"uuid": "c52ea54a-b949-48b1-928b-b497ecd97b30"}]}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:03:00.819985 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=14 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:03:00.820878 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=38 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:03:00.873631 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.network.neutron [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] validate_networks() for [('c52ea54a-b949-48b1-928b-b497ecd97b30', None, None, None, None, None)] {{(pid=130395) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 09:03:01.333695 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Flavor limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 09:03:01.334148 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Image limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 09:03:01.334623 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Flavor pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 09:03:01.335029 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Image pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 09:03:01.335437 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 09:03:01.345946 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:01.346422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:01.346857 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:01.347614 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:01.348069 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:01.348528 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:01.365866 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Getting quotas for project d0ccb79814f5432cb21c619e974fecd8. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 09:03:01.370180 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Getting quotas for user f452f27fc0d34f73aa22da2a09d54eb8 and project d0ccb79814f5432cb21c619e974fecd8. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 09:03:01.382215 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Going to run 1 instances... {{(pid=130395) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 09:03:01.384573 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bba331ba-faf5-4a56-81d9-6e84e9609665',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130395) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 09:03:01.431898 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR oslo.messaging._drivers.impl_rabbit [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [c1152517-df32-441b-8e44-ebce7596b7bc] AMQP server on 10.222.0.39:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 24 09:03:01.651451 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=2 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:03:01.653032 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=19 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:03:01.653799 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=90,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:03:01.658629 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b7264749-fb83-434d-8fd0-501a72f0b6b4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:01.659334 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b7264749-fb83-434d-8fd0-501a72f0b6b4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:01.672589 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b7264749-fb83-434d-8fd0-501a72f0b6b4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:01.673083 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b7264749-fb83-434d-8fd0-501a72f0b6b4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:01.673520 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b7264749-fb83-434d-8fd0-501a72f0b6b4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:01.674221 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b7264749-fb83-434d-8fd0-501a72f0b6b4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:01.674674 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b7264749-fb83-434d-8fd0-501a72f0b6b4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:01.675097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b7264749-fb83-434d-8fd0-501a72f0b6b4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:01.806073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b7264749-fb83-434d-8fd0-501a72f0b6b4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:01.815180 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b7264749-fb83-434d-8fd0-501a72f0b6b4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1593 microversion: 2.1 time: 0.165210 Oct 24 09:03:01.815774 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 526/1067] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:01 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1593 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 09:03:02.440664 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO oslo.messaging._drivers.impl_rabbit [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [c1152517-df32-441b-8e44-ebce7596b7bc] Reconnected to AMQP server on 10.222.0.39:5672 via [amqp] client with port 39088. Oct 24 09:03:02.444735 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-28b4eb1a-7b36-4d98-893c-45b38694d4de tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.642461 Oct 24 09:03:02.445414 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 542/1068] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 09:03:00 2022] POST /compute/v2.1/servers => generated 380 bytes in 1644 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 09:03:02.457362 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c9723dc0-956c-41b8-993b-6f501c386909 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:02.458170 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c9723dc0-956c-41b8-993b-6f501c386909 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:02.542062 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c9723dc0-956c-41b8-993b-6f501c386909 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:02.550951 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c9723dc0-956c-41b8-993b-6f501c386909 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1214 microversion: 2.1 time: 0.097339 Oct 24 09:03:02.551514 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 527/1069] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:02 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1214 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:03.576873 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-838f9155-f43b-440c-b46a-7c0bade220a8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:03.577612 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-838f9155-f43b-440c-b46a-7c0bade220a8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:03.585203 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-838f9155-f43b-440c-b46a-7c0bade220a8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:03.585973 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-838f9155-f43b-440c-b46a-7c0bade220a8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:03.586309 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-838f9155-f43b-440c-b46a-7c0bade220a8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:03.587078 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-838f9155-f43b-440c-b46a-7c0bade220a8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:03.587463 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-838f9155-f43b-440c-b46a-7c0bade220a8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:03.587890 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-838f9155-f43b-440c-b46a-7c0bade220a8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:03.690907 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-838f9155-f43b-440c-b46a-7c0bade220a8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:03.704755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-838f9155-f43b-440c-b46a-7c0bade220a8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1218 microversion: 2.1 time: 0.133250 Oct 24 09:03:03.706496 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 543/1070] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:03 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1218 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:03.837638 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-e7b159c5-ee2b-4677-8e80-282d219311dc req-8cafed0c-8fb1-4850-b87a-bc71be3b9679 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a7260920-be84-4a74-a18e-5b47d53e3db1", "tag": "b8947813-7d62-4e04-8aa8-8ba174d01def"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:03:03.851113 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-e7b159c5-ee2b-4677-8e80-282d219311dc req-8cafed0c-8fb1-4850-b87a-bc71be3b9679 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:03.851688 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-e7b159c5-ee2b-4677-8e80-282d219311dc req-8cafed0c-8fb1-4850-b87a-bc71be3b9679 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:03.852249 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-e7b159c5-ee2b-4677-8e80-282d219311dc req-8cafed0c-8fb1-4850-b87a-bc71be3b9679 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:03.864636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-e7b159c5-ee2b-4677-8e80-282d219311dc req-8cafed0c-8fb1-4850-b87a-bc71be3b9679 service nova] Creating event network-changed:b8947813-7d62-4e04-8aa8-8ba174d01def for instance a7260920-be84-4a74-a18e-5b47d53e3db1 on hv-cinder-maste Oct 24 09:03:03.870286 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-e7b159c5-ee2b-4677-8e80-282d219311dc req-8cafed0c-8fb1-4850-b87a-bc71be3b9679 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.240775 Oct 24 09:03:03.872003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 528/1071] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:03:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:03:04.640337 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-386fa824-3d24-4515-ace0-cf4488a2e596 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-188351888"}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:03:04.640905 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-386fa824-3d24-4515-ace0-cf4488a2e596 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:04.647034 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-386fa824-3d24-4515-ace0-cf4488a2e596 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:04.647516 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-386fa824-3d24-4515-ace0-cf4488a2e596 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:04.647955 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-386fa824-3d24-4515-ace0-cf4488a2e596 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:04.648690 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-386fa824-3d24-4515-ace0-cf4488a2e596 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:04.649141 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-386fa824-3d24-4515-ace0-cf4488a2e596 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:04.649565 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-386fa824-3d24-4515-ace0-cf4488a2e596 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:04.729919 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-dbac14a3-82bc-415e-8893-4f108c94884d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:04.731193 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-dbac14a3-82bc-415e-8893-4f108c94884d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:04.739556 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dbac14a3-82bc-415e-8893-4f108c94884d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:04.740048 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dbac14a3-82bc-415e-8893-4f108c94884d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:04.740513 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dbac14a3-82bc-415e-8893-4f108c94884d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:04.741215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dbac14a3-82bc-415e-8893-4f108c94884d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:04.741661 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dbac14a3-82bc-415e-8893-4f108c94884d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:04.742406 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-dbac14a3-82bc-415e-8893-4f108c94884d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:04.799013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.network.security_group_api [None req-386fa824-3d24-4515-ace0-cf4488a2e596 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Adding security group 84041028-15de-4e5b-bef7-4e848bffde5d to port b8947813-7d62-4e04-8aa8-8ba174d01def Oct 24 09:03:05.076365 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-dbac14a3-82bc-415e-8893-4f108c94884d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:05.085007 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-dbac14a3-82bc-415e-8893-4f108c94884d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1218 microversion: 2.1 time: 0.359659 Oct 24 09:03:05.085902 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 529/1072] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:04 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1218 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:05.388545 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-386fa824-3d24-4515-ace0-cf4488a2e596 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "POST /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/action" status: 202 len: 0 microversion: 2.1 time: 0.752099 Oct 24 09:03:05.389590 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 544/1073] 10.222.0.39 () {60 vars in 1278 bytes} [Mon Oct 24 09:03:04 2022] POST /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/action => generated 0 bytes in 754 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 24 09:03:05.397383 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6581ae03-747d-4bbb-b17d-c822e0f414cc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:05.398088 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6581ae03-747d-4bbb-b17d-c822e0f414cc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:05.404631 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6581ae03-747d-4bbb-b17d-c822e0f414cc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:05.405140 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6581ae03-747d-4bbb-b17d-c822e0f414cc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:05.405599 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6581ae03-747d-4bbb-b17d-c822e0f414cc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:05.406341 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6581ae03-747d-4bbb-b17d-c822e0f414cc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:05.406824 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6581ae03-747d-4bbb-b17d-c822e0f414cc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:05.407293 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6581ae03-747d-4bbb-b17d-c822e0f414cc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:05.560158 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6581ae03-747d-4bbb-b17d-c822e0f414cc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:05.568466 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6581ae03-747d-4bbb-b17d-c822e0f414cc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1639 microversion: 2.1 time: 0.175023 Oct 24 09:03:05.569039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 530/1074] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:05 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1639 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:05.887425 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-e6b15311-7e01-425c-8686-faee11ffb09c req-71d5846c-eeab-4579-aa7b-7ea08d0ab28d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a7260920-be84-4a74-a18e-5b47d53e3db1", "tag": "b8947813-7d62-4e04-8aa8-8ba174d01def"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:03:05.898007 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-e6b15311-7e01-425c-8686-faee11ffb09c req-71d5846c-eeab-4579-aa7b-7ea08d0ab28d service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:05.898474 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-e6b15311-7e01-425c-8686-faee11ffb09c req-71d5846c-eeab-4579-aa7b-7ea08d0ab28d service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:05.899221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-e6b15311-7e01-425c-8686-faee11ffb09c req-71d5846c-eeab-4579-aa7b-7ea08d0ab28d service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:05.911729 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-e6b15311-7e01-425c-8686-faee11ffb09c req-71d5846c-eeab-4579-aa7b-7ea08d0ab28d service nova] Creating event network-changed:b8947813-7d62-4e04-8aa8-8ba174d01def for instance a7260920-be84-4a74-a18e-5b47d53e3db1 on hv-cinder-maste Oct 24 09:03:05.916741 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-e6b15311-7e01-425c-8686-faee11ffb09c req-71d5846c-eeab-4579-aa7b-7ea08d0ab28d service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033772 Oct 24 09:03:05.917534 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 545/1075] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:03:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:03:06.109571 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0d1f1c8e-2312-46f5-8c79-289c3114e7b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:06.112263 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-0d1f1c8e-2312-46f5-8c79-289c3114e7b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:06.123495 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d1f1c8e-2312-46f5-8c79-289c3114e7b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:06.124491 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d1f1c8e-2312-46f5-8c79-289c3114e7b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:06.125454 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d1f1c8e-2312-46f5-8c79-289c3114e7b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:06.127469 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d1f1c8e-2312-46f5-8c79-289c3114e7b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:06.129910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d1f1c8e-2312-46f5-8c79-289c3114e7b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:06.130490 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d1f1c8e-2312-46f5-8c79-289c3114e7b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:06.222384 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0d1f1c8e-2312-46f5-8c79-289c3114e7b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:06.239224 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-0d1f1c8e-2312-46f5-8c79-289c3114e7b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1266 microversion: 2.1 time: 0.134486 Oct 24 09:03:06.239940 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 531/1076] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:06 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1266 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:07.265467 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a06c7c8f-4fa7-45ec-8d28-7057eb3eb2fa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:07.266200 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a06c7c8f-4fa7-45ec-8d28-7057eb3eb2fa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:07.272755 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a06c7c8f-4fa7-45ec-8d28-7057eb3eb2fa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:07.273238 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a06c7c8f-4fa7-45ec-8d28-7057eb3eb2fa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:07.273674 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a06c7c8f-4fa7-45ec-8d28-7057eb3eb2fa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:07.274375 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a06c7c8f-4fa7-45ec-8d28-7057eb3eb2fa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:07.274818 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a06c7c8f-4fa7-45ec-8d28-7057eb3eb2fa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:07.275242 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a06c7c8f-4fa7-45ec-8d28-7057eb3eb2fa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:07.342577 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a06c7c8f-4fa7-45ec-8d28-7057eb3eb2fa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:07.350387 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a06c7c8f-4fa7-45ec-8d28-7057eb3eb2fa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1272 microversion: 2.1 time: 0.090414 Oct 24 09:03:07.351127 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 546/1077] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:07 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1272 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:08.374876 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-43123780-49cd-4f09-80ee-6a2d9540365d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:08.375485 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-43123780-49cd-4f09-80ee-6a2d9540365d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:08.382116 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-43123780-49cd-4f09-80ee-6a2d9540365d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:08.382609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-43123780-49cd-4f09-80ee-6a2d9540365d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:08.383044 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-43123780-49cd-4f09-80ee-6a2d9540365d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:08.383747 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-43123780-49cd-4f09-80ee-6a2d9540365d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:08.384275 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-43123780-49cd-4f09-80ee-6a2d9540365d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:08.384654 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-43123780-49cd-4f09-80ee-6a2d9540365d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:08.515544 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-43123780-49cd-4f09-80ee-6a2d9540365d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:08.523196 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-43123780-49cd-4f09-80ee-6a2d9540365d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1314 microversion: 2.1 time: 0.152501 Oct 24 09:03:08.523763 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 532/1078] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:08 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1314 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:09.546049 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e060f62a-51ec-4174-b226-85535f536dc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:09.546799 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e060f62a-51ec-4174-b226-85535f536dc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:09.553272 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e060f62a-51ec-4174-b226-85535f536dc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:09.553748 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e060f62a-51ec-4174-b226-85535f536dc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:09.554181 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e060f62a-51ec-4174-b226-85535f536dc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:09.554867 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e060f62a-51ec-4174-b226-85535f536dc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:09.555316 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e060f62a-51ec-4174-b226-85535f536dc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:09.555739 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e060f62a-51ec-4174-b226-85535f536dc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:09.687395 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e060f62a-51ec-4174-b226-85535f536dc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:09.695369 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e060f62a-51ec-4174-b226-85535f536dc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1314 microversion: 2.1 time: 0.153230 Oct 24 09:03:09.695976 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 547/1079] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:09 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1314 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:10.719299 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7257376b-a945-4a52-8b2e-041cb9416f44 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:10.719847 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7257376b-a945-4a52-8b2e-041cb9416f44 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:10.726130 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7257376b-a945-4a52-8b2e-041cb9416f44 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:10.726605 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7257376b-a945-4a52-8b2e-041cb9416f44 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:10.727033 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7257376b-a945-4a52-8b2e-041cb9416f44 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:10.727722 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7257376b-a945-4a52-8b2e-041cb9416f44 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:10.728195 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7257376b-a945-4a52-8b2e-041cb9416f44 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:10.728614 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7257376b-a945-4a52-8b2e-041cb9416f44 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:10.861596 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7257376b-a945-4a52-8b2e-041cb9416f44 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:10.870531 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7257376b-a945-4a52-8b2e-041cb9416f44 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1314 microversion: 2.1 time: 0.155519 Oct 24 09:03:10.871262 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 533/1080] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:10 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:11.569726 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-66868130-251c-4d31-be1a-5e37bfd19264 req-2ca9202b-7c91-42ff-b919-2b495a1e28a7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "97470cae-f594-49e7-860f-cff890d836d0", "tag": "23784a70-8ff2-4f6d-940c-a6bb8c19d894"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:03:11.581477 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=2 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:03:11.582672 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=6 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:03:11.589107 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-66868130-251c-4d31-be1a-5e37bfd19264 req-2ca9202b-7c91-42ff-b919-2b495a1e28a7 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:11.590344 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-66868130-251c-4d31-be1a-5e37bfd19264 req-2ca9202b-7c91-42ff-b919-2b495a1e28a7 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:11.590999 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-66868130-251c-4d31-be1a-5e37bfd19264 req-2ca9202b-7c91-42ff-b919-2b495a1e28a7 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:11.604065 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-66868130-251c-4d31-be1a-5e37bfd19264 req-2ca9202b-7c91-42ff-b919-2b495a1e28a7 service nova] Creating event network-changed:23784a70-8ff2-4f6d-940c-a6bb8c19d894 for instance 97470cae-f594-49e7-860f-cff890d836d0 on hv-cinder-maste Oct 24 09:03:11.611941 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-66868130-251c-4d31-be1a-5e37bfd19264 req-2ca9202b-7c91-42ff-b919-2b495a1e28a7 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047620 Oct 24 09:03:11.612594 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 548/1081] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:03:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:03:11.895812 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-01264abd-5211-47c4-9c50-62fc5c1a4c59 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:11.896661 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-01264abd-5211-47c4-9c50-62fc5c1a4c59 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:11.904472 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-01264abd-5211-47c4-9c50-62fc5c1a4c59 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:11.905262 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-01264abd-5211-47c4-9c50-62fc5c1a4c59 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:11.905729 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-01264abd-5211-47c4-9c50-62fc5c1a4c59 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:11.906508 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-01264abd-5211-47c4-9c50-62fc5c1a4c59 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:11.907064 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-01264abd-5211-47c4-9c50-62fc5c1a4c59 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:11.907571 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-01264abd-5211-47c4-9c50-62fc5c1a4c59 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:12.046444 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-01264abd-5211-47c4-9c50-62fc5c1a4c59 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:12.054130 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-01264abd-5211-47c4-9c50-62fc5c1a4c59 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1314 microversion: 2.1 time: 0.162837 Oct 24 09:03:12.054692 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 534/1082] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:11 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1314 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:13.078950 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8c5fcc18-bfda-4651-8ca9-c6b5f06c4414 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:13.079660 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8c5fcc18-bfda-4651-8ca9-c6b5f06c4414 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:13.086529 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c5fcc18-bfda-4651-8ca9-c6b5f06c4414 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:13.087018 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c5fcc18-bfda-4651-8ca9-c6b5f06c4414 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:13.087458 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c5fcc18-bfda-4651-8ca9-c6b5f06c4414 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:13.088196 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c5fcc18-bfda-4651-8ca9-c6b5f06c4414 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:13.088654 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c5fcc18-bfda-4651-8ca9-c6b5f06c4414 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:13.089083 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c5fcc18-bfda-4651-8ca9-c6b5f06c4414 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:13.227557 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8c5fcc18-bfda-4651-8ca9-c6b5f06c4414 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:13.235345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8c5fcc18-bfda-4651-8ca9-c6b5f06c4414 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1314 microversion: 2.1 time: 0.161210 Oct 24 09:03:13.235913 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 549/1083] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:13 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1314 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:14.258922 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e78ea8c5-6ba9-4888-ac51-935448d2f16c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:14.259464 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e78ea8c5-6ba9-4888-ac51-935448d2f16c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:14.261357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=2 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:03:14.268217 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e78ea8c5-6ba9-4888-ac51-935448d2f16c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:14.268693 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e78ea8c5-6ba9-4888-ac51-935448d2f16c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:14.269120 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e78ea8c5-6ba9-4888-ac51-935448d2f16c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:14.269801 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e78ea8c5-6ba9-4888-ac51-935448d2f16c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:14.270249 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e78ea8c5-6ba9-4888-ac51-935448d2f16c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:14.270665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e78ea8c5-6ba9-4888-ac51-935448d2f16c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:14.400071 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e78ea8c5-6ba9-4888-ac51-935448d2f16c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:14.408255 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e78ea8c5-6ba9-4888-ac51-935448d2f16c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1314 microversion: 2.1 time: 0.153237 Oct 24 09:03:14.408818 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 535/1084] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:14 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1314 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:15.431226 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-2e68c117-3577-4f09-bc26-598d5423b3a2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:15.431823 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-2e68c117-3577-4f09-bc26-598d5423b3a2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:15.438483 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2e68c117-3577-4f09-bc26-598d5423b3a2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:15.438966 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2e68c117-3577-4f09-bc26-598d5423b3a2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:15.439409 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2e68c117-3577-4f09-bc26-598d5423b3a2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:15.440102 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2e68c117-3577-4f09-bc26-598d5423b3a2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:15.440578 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2e68c117-3577-4f09-bc26-598d5423b3a2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:15.441000 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2e68c117-3577-4f09-bc26-598d5423b3a2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:15.571833 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-2e68c117-3577-4f09-bc26-598d5423b3a2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:15.579475 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-2e68c117-3577-4f09-bc26-598d5423b3a2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1314 microversion: 2.1 time: 0.152170 Oct 24 09:03:15.580046 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 550/1085] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:15 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1314 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:16.603555 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b038783a-2aa3-43ce-965e-e7e6c07464ad tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:16.604179 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b038783a-2aa3-43ce-965e-e7e6c07464ad tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:16.611439 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b038783a-2aa3-43ce-965e-e7e6c07464ad tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:16.611925 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b038783a-2aa3-43ce-965e-e7e6c07464ad tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:16.612387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b038783a-2aa3-43ce-965e-e7e6c07464ad tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:16.613087 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b038783a-2aa3-43ce-965e-e7e6c07464ad tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:16.613535 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b038783a-2aa3-43ce-965e-e7e6c07464ad tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:16.613956 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b038783a-2aa3-43ce-965e-e7e6c07464ad tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:16.763027 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b038783a-2aa3-43ce-965e-e7e6c07464ad tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:16.770946 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b038783a-2aa3-43ce-965e-e7e6c07464ad tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1314 microversion: 2.1 time: 0.171190 Oct 24 09:03:16.771778 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 536/1086] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:16 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1314 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:17.801134 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-27a77c0b-d577-4af7-a839-92a557922b6e tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:17.801775 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-27a77c0b-d577-4af7-a839-92a557922b6e tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:17.808026 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-27a77c0b-d577-4af7-a839-92a557922b6e tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:17.809169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-27a77c0b-d577-4af7-a839-92a557922b6e tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:17.810049 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-27a77c0b-d577-4af7-a839-92a557922b6e tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:17.810742 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-27a77c0b-d577-4af7-a839-92a557922b6e tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:17.811199 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-27a77c0b-d577-4af7-a839-92a557922b6e tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:17.811620 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-27a77c0b-d577-4af7-a839-92a557922b6e tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:17.943259 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-27a77c0b-d577-4af7-a839-92a557922b6e tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:17.951520 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-27a77c0b-d577-4af7-a839-92a557922b6e tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1314 microversion: 2.1 time: 0.154238 Oct 24 09:03:17.952084 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 551/1087] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:17 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1314 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:18.979370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b9b29752-3e16-434d-a8a0-b1245c3e7028 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:18.979993 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b9b29752-3e16-434d-a8a0-b1245c3e7028 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:18.987105 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b9b29752-3e16-434d-a8a0-b1245c3e7028 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:18.987631 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b9b29752-3e16-434d-a8a0-b1245c3e7028 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:18.988068 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b9b29752-3e16-434d-a8a0-b1245c3e7028 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:18.988855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b9b29752-3e16-434d-a8a0-b1245c3e7028 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:18.989340 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b9b29752-3e16-434d-a8a0-b1245c3e7028 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:18.989814 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b9b29752-3e16-434d-a8a0-b1245c3e7028 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:19.126638 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-b9b29752-3e16-434d-a8a0-b1245c3e7028 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:19.135764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b9b29752-3e16-434d-a8a0-b1245c3e7028 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1314 microversion: 2.1 time: 0.160970 Oct 24 09:03:19.136763 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 537/1088] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:18 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1314 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:20.159857 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d412eed5-b0a3-4b41-8a39-e9e2f44b7632 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:20.160503 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d412eed5-b0a3-4b41-8a39-e9e2f44b7632 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:20.167029 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d412eed5-b0a3-4b41-8a39-e9e2f44b7632 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:20.167782 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d412eed5-b0a3-4b41-8a39-e9e2f44b7632 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:20.168629 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d412eed5-b0a3-4b41-8a39-e9e2f44b7632 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:20.169769 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d412eed5-b0a3-4b41-8a39-e9e2f44b7632 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:20.170576 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d412eed5-b0a3-4b41-8a39-e9e2f44b7632 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:20.171359 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d412eed5-b0a3-4b41-8a39-e9e2f44b7632 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:20.189922 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b9006d76-b5f9-44fa-b03e-2165074a4af1 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:20.190550 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b9006d76-b5f9-44fa-b03e-2165074a4af1 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:20.198422 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b9006d76-b5f9-44fa-b03e-2165074a4af1 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:20.199026 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b9006d76-b5f9-44fa-b03e-2165074a4af1 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:20.201020 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b9006d76-b5f9-44fa-b03e-2165074a4af1 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:20.201862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b9006d76-b5f9-44fa-b03e-2165074a4af1 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:20.202418 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b9006d76-b5f9-44fa-b03e-2165074a4af1 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:20.203566 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b9006d76-b5f9-44fa-b03e-2165074a4af1 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:20.235048 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b9006d76-b5f9-44fa-b03e-2165074a4af1 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Going to try to terminate instance {{(pid=130396) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 09:03:20.326766 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b9006d76-b5f9-44fa-b03e-2165074a4af1 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "DELETE /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 204 len: 0 microversion: 2.1 time: 0.142160 Oct 24 09:03:20.327769 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 538/1089] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 09:03:20 2022] DELETE /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 09:03:20.343231 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-89222373-078a-4e80-bc5f-55df4803e1b4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:20.344103 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-89222373-078a-4e80-bc5f-55df4803e1b4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:20.351671 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-89222373-078a-4e80-bc5f-55df4803e1b4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:20.352444 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-89222373-078a-4e80-bc5f-55df4803e1b4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:20.353100 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-89222373-078a-4e80-bc5f-55df4803e1b4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:20.354139 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-89222373-078a-4e80-bc5f-55df4803e1b4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:20.354821 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-89222373-078a-4e80-bc5f-55df4803e1b4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:20.355435 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-89222373-078a-4e80-bc5f-55df4803e1b4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:20.477721 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d412eed5-b0a3-4b41-8a39-e9e2f44b7632 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:20.490499 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d412eed5-b0a3-4b41-8a39-e9e2f44b7632 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1314 microversion: 2.1 time: 0.334461 Oct 24 09:03:20.491644 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 552/1090] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:20 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1314 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:20.546996 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-ca0497ea-aa8a-4bfa-8cbf-75773f24e63c req-53d51835-075a-4ea0-8dec-b7886b1b97af service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "97470cae-f594-49e7-860f-cff890d836d0", "name": "network-vif-plugged", "status": "completed", "tag": "23784a70-8ff2-4f6d-940c-a6bb8c19d894"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:03:20.557443 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-ca0497ea-aa8a-4bfa-8cbf-75773f24e63c req-53d51835-075a-4ea0-8dec-b7886b1b97af service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:20.558122 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-ca0497ea-aa8a-4bfa-8cbf-75773f24e63c req-53d51835-075a-4ea0-8dec-b7886b1b97af service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:20.559153 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-ca0497ea-aa8a-4bfa-8cbf-75773f24e63c req-53d51835-075a-4ea0-8dec-b7886b1b97af service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:20.572066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-ca0497ea-aa8a-4bfa-8cbf-75773f24e63c req-53d51835-075a-4ea0-8dec-b7886b1b97af service nova] Creating event network-vif-plugged:23784a70-8ff2-4f6d-940c-a6bb8c19d894 for instance 97470cae-f594-49e7-860f-cff890d836d0 on hv-cinder-maste Oct 24 09:03:20.573883 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-89222373-078a-4e80-bc5f-55df4803e1b4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:20.578483 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-ca0497ea-aa8a-4bfa-8cbf-75773f24e63c req-53d51835-075a-4ea0-8dec-b7886b1b97af service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036417 Oct 24 09:03:20.579351 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 553/1091] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:03:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:03:20.583748 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-89222373-078a-4e80-bc5f-55df4803e1b4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1576 microversion: 2.1 time: 0.251768 Oct 24 09:03:20.585173 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 539/1092] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:20 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1576 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 09:03:21.515509 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-90a68a67-e38b-408e-8a1a-2272df0eabdc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:21.516229 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-90a68a67-e38b-408e-8a1a-2272df0eabdc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:21.522374 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-90a68a67-e38b-408e-8a1a-2272df0eabdc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:21.522861 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-90a68a67-e38b-408e-8a1a-2272df0eabdc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:21.523289 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-90a68a67-e38b-408e-8a1a-2272df0eabdc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:21.523969 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-90a68a67-e38b-408e-8a1a-2272df0eabdc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:21.524446 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-90a68a67-e38b-408e-8a1a-2272df0eabdc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:21.524868 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-90a68a67-e38b-408e-8a1a-2272df0eabdc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:21.609881 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-51044da3-162d-4c16-996a-537eab1f15ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:21.610499 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-51044da3-162d-4c16-996a-537eab1f15ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:21.617524 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-51044da3-162d-4c16-996a-537eab1f15ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:21.618015 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-51044da3-162d-4c16-996a-537eab1f15ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:21.618719 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-51044da3-162d-4c16-996a-537eab1f15ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:21.619424 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-51044da3-162d-4c16-996a-537eab1f15ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:21.619919 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-51044da3-162d-4c16-996a-537eab1f15ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:21.620387 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-51044da3-162d-4c16-996a-537eab1f15ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:21.677843 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-90a68a67-e38b-408e-8a1a-2272df0eabdc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:21.687105 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-90a68a67-e38b-408e-8a1a-2272df0eabdc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1314 microversion: 2.1 time: 0.176643 Oct 24 09:03:21.687724 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 554/1093] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:21 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1314 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:21.771694 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-51044da3-162d-4c16-996a-537eab1f15ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:21.780361 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-51044da3-162d-4c16-996a-537eab1f15ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1576 microversion: 2.1 time: 0.175387 Oct 24 09:03:21.780968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 540/1094] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:21 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1576 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 09:03:22.710966 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-865a07cc-31f1-4207-be60-5e2181fc9866 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:22.711506 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-865a07cc-31f1-4207-be60-5e2181fc9866 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:22.718675 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-865a07cc-31f1-4207-be60-5e2181fc9866 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:22.719169 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-865a07cc-31f1-4207-be60-5e2181fc9866 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:22.719620 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-865a07cc-31f1-4207-be60-5e2181fc9866 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:22.720366 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-865a07cc-31f1-4207-be60-5e2181fc9866 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:22.720828 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-865a07cc-31f1-4207-be60-5e2181fc9866 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:22.721262 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-865a07cc-31f1-4207-be60-5e2181fc9866 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:22.802990 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-d7ecb01d-1fd6-43f2-94b9-5936b6dd785e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:22.803607 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-d7ecb01d-1fd6-43f2-94b9-5936b6dd785e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:22.811052 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7ecb01d-1fd6-43f2-94b9-5936b6dd785e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:22.811564 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7ecb01d-1fd6-43f2-94b9-5936b6dd785e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:22.811998 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7ecb01d-1fd6-43f2-94b9-5936b6dd785e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:22.812748 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7ecb01d-1fd6-43f2-94b9-5936b6dd785e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:22.813197 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7ecb01d-1fd6-43f2-94b9-5936b6dd785e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:22.813622 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7ecb01d-1fd6-43f2-94b9-5936b6dd785e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:22.863463 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-865a07cc-31f1-4207-be60-5e2181fc9866 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:22.872188 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-865a07cc-31f1-4207-be60-5e2181fc9866 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1314 microversion: 2.1 time: 0.165161 Oct 24 09:03:22.873141 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 555/1095] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:22 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1314 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:22.955678 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d7ecb01d-1fd6-43f2-94b9-5936b6dd785e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:22.963340 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-d7ecb01d-1fd6-43f2-94b9-5936b6dd785e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1576 microversion: 2.1 time: 0.164328 Oct 24 09:03:22.963836 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 541/1096] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:22 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1576 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 09:03:23.899601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-00d9c338-8969-49d1-8848-1cfe462a9d3b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:23.900010 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-00d9c338-8969-49d1-8848-1cfe462a9d3b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:23.925552 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-00d9c338-8969-49d1-8848-1cfe462a9d3b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:23.928255 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-00d9c338-8969-49d1-8848-1cfe462a9d3b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:23.929618 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-00d9c338-8969-49d1-8848-1cfe462a9d3b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:23.932966 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-00d9c338-8969-49d1-8848-1cfe462a9d3b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:23.934050 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-00d9c338-8969-49d1-8848-1cfe462a9d3b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:23.934790 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-00d9c338-8969-49d1-8848-1cfe462a9d3b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:23.984299 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-cdd023c6-abee-4007-98c4-9d70641278d7 req-b8107718-52e1-42d7-a5c1-8f902d245ab0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f2c4ad60-dd40-42b5-8728-8a414e8d7644", "name": "network-vif-deleted", "tag": "ea79d810-db9b-4fba-9d68-f8b0560784ee"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:03:24.006144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-cdd023c6-abee-4007-98c4-9d70641278d7 req-b8107718-52e1-42d7-a5c1-8f902d245ab0 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:24.007108 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-cdd023c6-abee-4007-98c4-9d70641278d7 req-b8107718-52e1-42d7-a5c1-8f902d245ab0 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:24.007764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-cdd023c6-abee-4007-98c4-9d70641278d7 req-b8107718-52e1-42d7-a5c1-8f902d245ab0 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:24.023155 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-cdd023c6-abee-4007-98c4-9d70641278d7 req-b8107718-52e1-42d7-a5c1-8f902d245ab0 service nova] Creating event network-vif-deleted:ea79d810-db9b-4fba-9d68-f8b0560784ee for instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 on hv-cinder-maste Oct 24 09:03:24.035499 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-cdd023c6-abee-4007-98c4-9d70641278d7 req-b8107718-52e1-42d7-a5c1-8f902d245ab0 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055162 Oct 24 09:03:24.036527 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 542/1097] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:03:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 09:03:24.056241 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-662c206e-1983-474d-afb7-e50b6ef829ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:24.064080 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-662c206e-1983-474d-afb7-e50b6ef829ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:24.080032 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-662c206e-1983-474d-afb7-e50b6ef829ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:24.088951 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-662c206e-1983-474d-afb7-e50b6ef829ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:24.089743 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-662c206e-1983-474d-afb7-e50b6ef829ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:24.091711 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-662c206e-1983-474d-afb7-e50b6ef829ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:24.092469 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-662c206e-1983-474d-afb7-e50b6ef829ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:24.093156 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-662c206e-1983-474d-afb7-e50b6ef829ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:24.098248 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:03:24.244391 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-662c206e-1983-474d-afb7-e50b6ef829ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:24.255448 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-662c206e-1983-474d-afb7-e50b6ef829ae tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 200 len: 1225 microversion: 2.1 time: 0.203360 Oct 24 09:03:24.256250 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 543/1098] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:24 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 1225 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:24.269791 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-00d9c338-8969-49d1-8848-1cfe462a9d3b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:24.279679 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-00d9c338-8969-49d1-8848-1cfe462a9d3b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1499 microversion: 2.1 time: 0.385916 Oct 24 09:03:24.283054 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 556/1099] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:23 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1499 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:24.869398 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-cb34727b-649e-4ccd-866f-19fb4f3a93f8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:24.869910 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-cb34727b-649e-4ccd-866f-19fb4f3a93f8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:24.876914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cb34727b-649e-4ccd-866f-19fb4f3a93f8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:24.877418 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cb34727b-649e-4ccd-866f-19fb4f3a93f8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:24.877861 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cb34727b-649e-4ccd-866f-19fb4f3a93f8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:24.878574 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cb34727b-649e-4ccd-866f-19fb4f3a93f8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:24.879026 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cb34727b-649e-4ccd-866f-19fb4f3a93f8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:24.879452 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-cb34727b-649e-4ccd-866f-19fb4f3a93f8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:24.910931 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-cb34727b-649e-4ccd-866f-19fb4f3a93f8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Going to try to terminate instance {{(pid=130396) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 09:03:25.021032 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-cb34727b-649e-4ccd-866f-19fb4f3a93f8 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "DELETE /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 204 len: 0 microversion: 2.1 time: 0.161325 Oct 24 09:03:25.021614 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 544/1100] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 09:03:24 2022] DELETE /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 09:03:25.030461 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-947a2d4d-1ba4-49be-b5f2-3f16e486ccc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:25.031102 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-947a2d4d-1ba4-49be-b5f2-3f16e486ccc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:25.037634 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-947a2d4d-1ba4-49be-b5f2-3f16e486ccc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:25.038127 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-947a2d4d-1ba4-49be-b5f2-3f16e486ccc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:25.038577 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-947a2d4d-1ba4-49be-b5f2-3f16e486ccc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:25.039455 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-947a2d4d-1ba4-49be-b5f2-3f16e486ccc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:25.039924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-947a2d4d-1ba4-49be-b5f2-3f16e486ccc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:25.040452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-947a2d4d-1ba4-49be-b5f2-3f16e486ccc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:25.233771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-947a2d4d-1ba4-49be-b5f2-3f16e486ccc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:25.259874 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-947a2d4d-1ba4-49be-b5f2-3f16e486ccc6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1505 microversion: 2.1 time: 0.233762 Oct 24 09:03:25.261100 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 557/1101] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:25 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1505 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:25.279527 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8d3577ed-b168-4916-ad53-9312d5756724 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:25.280163 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8d3577ed-b168-4916-ad53-9312d5756724 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:25.292753 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8d3577ed-b168-4916-ad53-9312d5756724 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:25.293547 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8d3577ed-b168-4916-ad53-9312d5756724 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:25.294220 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8d3577ed-b168-4916-ad53-9312d5756724 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:25.295081 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8d3577ed-b168-4916-ad53-9312d5756724 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:25.295688 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8d3577ed-b168-4916-ad53-9312d5756724 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:25.296306 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8d3577ed-b168-4916-ad53-9312d5756724 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:25.311902 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-8d3577ed-b168-4916-ad53-9312d5756724 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] HTTP exception thrown: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8d3577ed-b168-4916-ad53-9312d5756724 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Request method failure captured: Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 HTTP/1.1 Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlQLoEuIeuhEpW8bsRGnfBcSL-MruEuqIHx01hIOb-lCoWsb18wEa-RL0_ZfZgzNX7tLN6J7zRPamBYUN5Q2h22RCUYnOaAbGpXYsIe862ctqswbh9eXLbV__Gfvp2TY03EtBVsWLXdYyHLJmVeYSUmfNZChusOQVadnrf3HbMtegdk Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 30742e7b914243df8dd120079560a3de Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-TestVolumeBootPattern-2039901937 Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 09:03:25.314182 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 09:03:25.316003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/30742e7b914243df8dd120079560a3de"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_30742e7b914243df8dd120079560a3de", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/30742e7b914243df8dd120079560a3de"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/30742e7b914243df8dd120079560a3de"}]}] Oct 24 09:03:25.316003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-TestVolumeBootPattern-2039901937 Oct 24 09:03:25.316003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 30742e7b914243df8dd120079560a3de Oct 24 09:03:25.316003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-TestVolumeBootPattern-2039901937 Oct 24 09:03:25.316003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-TestVolumeBootPattern-2039901937-project Oct 24 09:03:25.316003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 09:03:25.316003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 09:03:25.316003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 3cddd27c37514e74a38430c46cef16fa Oct 24 09:03:25.316003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-TestVolumeBootPattern-2039901937-project Oct 24 09:03:25.316003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 09:03:25.316003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. Oct 24 09:03:25.316003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': 'f2c4ad60-dd40-42b5-8728-8a414e8d7644'} Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:03:25.317025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. Oct 24 09:03:25.318914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:03:25.320108 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8d3577ed-b168-4916-ad53-9312d5756724 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Returning 404 to user: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:03:25.320108 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8d3577ed-b168-4916-ad53-9312d5756724 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 404 len: 111 microversion: 2.1 time: 0.043600 Oct 24 09:03:25.320108 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 545/1102] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:25 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:03:25.434387 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-430508639", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-593019647", "security_groups": [{"name": "tempest-secgroup-smoke-1512102953"}], "block_device_mapping_v2": [{"uuid": "c0807ccc-7846-4c0d-8cac-7c6f691a0ba9", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "15551789-d406-4f91-85fe-9eee40e4ab51"}]}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:03:25.556227 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.network.neutron [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] validate_networks() for [('15551789-d406-4f91-85fe-9eee40e4ab51', None, None, None, None, None)] {{(pid=130395) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 09:03:25.947043 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Flavor limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 09:03:25.947534 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Image limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 09:03:25.948225 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Flavor pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 09:03:25.948641 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Image pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 09:03:25.949061 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 09:03:25.957970 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:25.958460 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:25.958899 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:25.959724 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:25.960211 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:25.960646 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:25.978401 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Getting quotas for project 30742e7b914243df8dd120079560a3de. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 09:03:25.982505 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Getting quotas for user 3cddd27c37514e74a38430c46cef16fa and project 30742e7b914243df8dd120079560a3de. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 09:03:26.040567 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Going to run 1 instances... {{(pid=130395) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 09:03:26.042680 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='c0807ccc-7846-4c0d-8cac-7c6f691a0ba9',volume_size=None,volume_type=None)] {{(pid=130395) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 09:03:26.196355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ed56eb41-5895-48d7-ad05-4c78034c8038 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.766565 Oct 24 09:03:26.197013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 558/1103] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 09:03:25 2022] POST /compute/v2.1/servers => generated 406 bytes in 768 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 09:03:26.208787 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7382e8c8-0901-4e67-aec9-3c5bef6fe7fc tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:26.210010 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7382e8c8-0901-4e67-aec9-3c5bef6fe7fc tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:26.288224 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a1549b20-0466-4c82-950d-a5972154e38b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:26.289458 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a1549b20-0466-4c82-950d-a5972154e38b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:26.290020 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7382e8c8-0901-4e67-aec9-3c5bef6fe7fc tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:26.297668 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a1549b20-0466-4c82-950d-a5972154e38b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:26.298038 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a1549b20-0466-4c82-950d-a5972154e38b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:26.298803 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a1549b20-0466-4c82-950d-a5972154e38b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:26.300520 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a1549b20-0466-4c82-950d-a5972154e38b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:26.301317 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a1549b20-0466-4c82-950d-a5972154e38b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:26.302051 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a1549b20-0466-4c82-950d-a5972154e38b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:26.310762 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7382e8c8-0901-4e67-aec9-3c5bef6fe7fc tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1093 microversion: 2.1 time: 0.105571 Oct 24 09:03:26.313448 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 546/1104] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:26 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1093 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:26.462528 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a1549b20-0466-4c82-950d-a5972154e38b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:26.474669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a1549b20-0466-4c82-950d-a5972154e38b tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1505 microversion: 2.1 time: 0.192167 Oct 24 09:03:26.475704 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 559/1105] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:26 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1505 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:27.335189 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-4368b69f-da16-4932-8196-78cd96b894aa tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:27.335806 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-4368b69f-da16-4932-8196-78cd96b894aa tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:27.344926 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4368b69f-da16-4932-8196-78cd96b894aa tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:27.345636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4368b69f-da16-4932-8196-78cd96b894aa tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:27.346470 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4368b69f-da16-4932-8196-78cd96b894aa tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:27.347203 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4368b69f-da16-4932-8196-78cd96b894aa tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:27.347879 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4368b69f-da16-4932-8196-78cd96b894aa tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:27.348340 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-4368b69f-da16-4932-8196-78cd96b894aa tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:27.502470 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d360c2b3-920e-428c-8ed9-129ee535b6b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:27.503088 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d360c2b3-920e-428c-8ed9-129ee535b6b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:27.510696 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d360c2b3-920e-428c-8ed9-129ee535b6b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:27.511238 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d360c2b3-920e-428c-8ed9-129ee535b6b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:27.511764 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d360c2b3-920e-428c-8ed9-129ee535b6b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:27.512736 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d360c2b3-920e-428c-8ed9-129ee535b6b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:27.513196 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d360c2b3-920e-428c-8ed9-129ee535b6b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:27.513643 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d360c2b3-920e-428c-8ed9-129ee535b6b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:27.517906 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-4368b69f-da16-4932-8196-78cd96b894aa tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:27.530723 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-4368b69f-da16-4932-8196-78cd96b894aa tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1199 microversion: 2.1 time: 0.199370 Oct 24 09:03:27.550094 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 547/1106] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:27 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1199 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:27.556345 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-5fd83eb7-c485-45df-a6b6-3c1e77e017bd req-24e5c2a8-08d6-4890-8b87-8ab6caca2372 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "97470cae-f594-49e7-860f-cff890d836d0", "name": "network-vif-deleted", "tag": "23784a70-8ff2-4f6d-940c-a6bb8c19d894"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:03:27.568193 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-5fd83eb7-c485-45df-a6b6-3c1e77e017bd req-24e5c2a8-08d6-4890-8b87-8ab6caca2372 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:27.568807 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-5fd83eb7-c485-45df-a6b6-3c1e77e017bd req-24e5c2a8-08d6-4890-8b87-8ab6caca2372 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:27.569453 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-5fd83eb7-c485-45df-a6b6-3c1e77e017bd req-24e5c2a8-08d6-4890-8b87-8ab6caca2372 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:27.584702 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-5fd83eb7-c485-45df-a6b6-3c1e77e017bd req-24e5c2a8-08d6-4890-8b87-8ab6caca2372 service nova] Creating event network-vif-deleted:23784a70-8ff2-4f6d-940c-a6bb8c19d894 for instance 97470cae-f594-49e7-860f-cff890d836d0 on hv-cinder-maste Oct 24 09:03:27.591832 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-5fd83eb7-c485-45df-a6b6-3c1e77e017bd req-24e5c2a8-08d6-4890-8b87-8ab6caca2372 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037404 Oct 24 09:03:27.592882 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 548/1107] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:03:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:03:27.627703 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d360c2b3-920e-428c-8ed9-129ee535b6b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:27.638144 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d360c2b3-920e-428c-8ed9-129ee535b6b7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 200 len: 1463 microversion: 2.1 time: 0.139832 Oct 24 09:03:27.639044 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 560/1108] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:27 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 1463 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:28.565571 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-50516cb2-ebc0-4884-8b71-9f1066c9522f tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:28.566671 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-50516cb2-ebc0-4884-8b71-9f1066c9522f tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:28.582713 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-50516cb2-ebc0-4884-8b71-9f1066c9522f tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:28.583288 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-50516cb2-ebc0-4884-8b71-9f1066c9522f tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:28.583801 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-50516cb2-ebc0-4884-8b71-9f1066c9522f tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:28.585195 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-50516cb2-ebc0-4884-8b71-9f1066c9522f tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:28.586036 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-50516cb2-ebc0-4884-8b71-9f1066c9522f tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:28.588057 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-50516cb2-ebc0-4884-8b71-9f1066c9522f tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:28.662932 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c39d9bc3-085f-4f54-a2a4-1a63b78f2933 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:28.663713 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c39d9bc3-085f-4f54-a2a4-1a63b78f2933 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 97470cae-f594-49e7-860f-cff890d836d0] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:28.671387 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c39d9bc3-085f-4f54-a2a4-1a63b78f2933 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:28.672089 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c39d9bc3-085f-4f54-a2a4-1a63b78f2933 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:28.672788 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c39d9bc3-085f-4f54-a2a4-1a63b78f2933 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:28.673754 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c39d9bc3-085f-4f54-a2a4-1a63b78f2933 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:28.674434 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c39d9bc3-085f-4f54-a2a4-1a63b78f2933 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:28.675085 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c39d9bc3-085f-4f54-a2a4-1a63b78f2933 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:28.690049 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-c39d9bc3-085f-4f54-a2a4-1a63b78f2933 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] HTTP exception thrown: Instance 97470cae-f594-49e7-860f-cff890d836d0 could not be found. Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c39d9bc3-085f-4f54-a2a4-1a63b78f2933 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Request method failure captured: Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 HTTP/1.1 Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlS_ru2bHllLLO725iFN5fiACg8PQIsNVjiesAdzAyp0HjdBT5pVV_4eVQ3005xtqUBAbB3XSMIABOqBsnXCCzrEMcbPGiQnhbGz2k_vYJDZeOv0GDVi93VJ7Ne3BGBQCU9kRzxSKjgavwF0nTNzLX5CrZwV-_qnkZMIhgXflopg3h0 Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: d0ccb79814f5432cb21c619e974fecd8 Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-VolumesActionsTest-1529311557 Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:03:28.692810 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:03:28.694563 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/d0ccb79814f5432cb21c619e974fecd8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_d0ccb79814f5432cb21c619e974fecd8", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/d0ccb79814f5432cb21c619e974fecd8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/d0ccb79814f5432cb21c619e974fecd8"}]}] Oct 24 09:03:28.694563 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-VolumesActionsTest-1529311557 Oct 24 09:03:28.694563 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: d0ccb79814f5432cb21c619e974fecd8 Oct 24 09:03:28.694563 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-VolumesActionsTest-1529311557 Oct 24 09:03:28.694563 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-VolumesActionsTest-1529311557-project Oct 24 09:03:28.694563 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:03:28.694563 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:03:28.694563 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: f452f27fc0d34f73aa22da2a09d54eb8 Oct 24 09:03:28.694563 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-VolumesActionsTest-1529311557-project Oct 24 09:03:28.694563 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 09:03:28.694563 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance 97470cae-f594-49e7-860f-cff890d836d0 could not be found. Oct 24 09:03:28.694563 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': '97470cae-f594-49e7-860f-cff890d836d0'} Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 97470cae-f594-49e7-860f-cff890d836d0 could not be found. Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:03:28.695589 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 97470cae-f594-49e7-860f-cff890d836d0 could not be found. Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 97470cae-f594-49e7-860f-cff890d836d0 could not be found. Oct 24 09:03:28.697345 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:03:28.700614 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c39d9bc3-085f-4f54-a2a4-1a63b78f2933 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Returning 404 to user: Instance 97470cae-f594-49e7-860f-cff890d836d0 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:03:28.701954 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c39d9bc3-085f-4f54-a2a4-1a63b78f2933 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0" status: 404 len: 111 microversion: 2.1 time: 0.042819 Oct 24 09:03:28.702983 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 561/1109] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:28 2022] GET /compute/v2.1/servers/97470cae-f594-49e7-860f-cff890d836d0 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 24 09:03:28.776474 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-50516cb2-ebc0-4884-8b71-9f1066c9522f tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:28.787567 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-50516cb2-ebc0-4884-8b71-9f1066c9522f tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1265 microversion: 2.1 time: 0.231060 Oct 24 09:03:28.789016 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 549/1110] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:28 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1265 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:28.822478 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-812354972", "imageRef": "bba331ba-faf5-4a56-81d9-6e84e9609665", "flavorRef": "452", "networks": [{"uuid": "c52ea54a-b949-48b1-928b-b497ecd97b30"}]}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:03:28.881197 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.network.neutron [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] validate_networks() for [('c52ea54a-b949-48b1-928b-b497ecd97b30', None, None, None, None, None)] {{(pid=130395) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 09:03:29.134864 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-6b1de042-7a39-496d-ae88-2b03d57706ae req-a613954f-3f0a-46bd-b2c2-e52b9d128d4c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1812c1d4-2552-4a5e-9e55-42c7eec001cd", "tag": "f4ba7cc3-9768-4fde-9952-2e905e5b30e5"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:03:29.159797 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-6b1de042-7a39-496d-ae88-2b03d57706ae req-a613954f-3f0a-46bd-b2c2-e52b9d128d4c service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:29.159972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-6b1de042-7a39-496d-ae88-2b03d57706ae req-a613954f-3f0a-46bd-b2c2-e52b9d128d4c service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:29.160401 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-6b1de042-7a39-496d-ae88-2b03d57706ae req-a613954f-3f0a-46bd-b2c2-e52b9d128d4c service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:29.188443 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-6b1de042-7a39-496d-ae88-2b03d57706ae req-a613954f-3f0a-46bd-b2c2-e52b9d128d4c service nova] Creating event network-changed:f4ba7cc3-9768-4fde-9952-2e905e5b30e5 for instance 1812c1d4-2552-4a5e-9e55-42c7eec001cd on hv-cinder-maste Oct 24 09:03:29.194641 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-6b1de042-7a39-496d-ae88-2b03d57706ae req-a613954f-3f0a-46bd-b2c2-e52b9d128d4c service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.064791 Oct 24 09:03:29.195908 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 550/1111] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:03:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 09:03:29.298801 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Flavor limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 09:03:29.299230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Image limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 09:03:29.299769 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Flavor pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 09:03:29.300213 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Image pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 09:03:29.300679 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 09:03:29.309612 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:29.310095 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:29.310531 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:29.311276 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:29.311701 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:29.312347 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:29.329636 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Getting quotas for project d0ccb79814f5432cb21c619e974fecd8. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 09:03:29.333419 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Getting quotas for user f452f27fc0d34f73aa22da2a09d54eb8 and project d0ccb79814f5432cb21c619e974fecd8. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 09:03:29.345413 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Going to run 1 instances... {{(pid=130395) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 09:03:29.347621 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bba331ba-faf5-4a56-81d9-6e84e9609665',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130395) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 09:03:29.397465 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-dc6786c1-7b39-4e69-8669-69590e6ae7d2 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.578917 Oct 24 09:03:29.398256 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 562/1112] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 09:03:28 2022] POST /compute/v2.1/servers => generated 380 bytes in 580 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 09:03:29.410041 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9556b56d-5ed8-4306-b661-af0fdac20663 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:29.411064 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9556b56d-5ed8-4306-b661-af0fdac20663 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:29.473731 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9556b56d-5ed8-4306-b661-af0fdac20663 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:29.482300 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9556b56d-5ed8-4306-b661-af0fdac20663 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1213 microversion: 2.1 time: 0.076350 Oct 24 09:03:29.483114 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 551/1113] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:29 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1213 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:29.814470 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-2d183177-6ed9-4271-9eb3-a39e4c897846 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:29.815218 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-2d183177-6ed9-4271-9eb3-a39e4c897846 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:29.823943 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2d183177-6ed9-4271-9eb3-a39e4c897846 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:29.825050 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2d183177-6ed9-4271-9eb3-a39e4c897846 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:29.825601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2d183177-6ed9-4271-9eb3-a39e4c897846 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:29.826749 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2d183177-6ed9-4271-9eb3-a39e4c897846 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:29.827356 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2d183177-6ed9-4271-9eb3-a39e4c897846 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:29.828213 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-2d183177-6ed9-4271-9eb3-a39e4c897846 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:29.849999 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=100 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:03:29.967670 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-2d183177-6ed9-4271-9eb3-a39e4c897846 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:29.975355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-2d183177-6ed9-4271-9eb3-a39e4c897846 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1265 microversion: 2.1 time: 0.165541 Oct 24 09:03:29.975918 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 563/1114] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:29 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1265 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:30.505271 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-1c066e89-a5f2-4089-9e7d-f40f29de243a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:30.505911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-1c066e89-a5f2-4089-9e7d-f40f29de243a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:30.513953 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1c066e89-a5f2-4089-9e7d-f40f29de243a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:30.514474 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1c066e89-a5f2-4089-9e7d-f40f29de243a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:30.515131 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1c066e89-a5f2-4089-9e7d-f40f29de243a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:30.515900 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1c066e89-a5f2-4089-9e7d-f40f29de243a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:30.516442 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1c066e89-a5f2-4089-9e7d-f40f29de243a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:30.516905 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-1c066e89-a5f2-4089-9e7d-f40f29de243a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:30.584992 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-1c066e89-a5f2-4089-9e7d-f40f29de243a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:30.593126 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-1c066e89-a5f2-4089-9e7d-f40f29de243a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1217 microversion: 2.1 time: 0.091630 Oct 24 09:03:30.593721 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 552/1115] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:30 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1217 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:30.999553 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-23761faf-a578-45d6-aa99-0207e28446e4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:31.000204 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-23761faf-a578-45d6-aa99-0207e28446e4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:31.002474 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=80,nova_api:INSERT=9 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:03:31.007608 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-23761faf-a578-45d6-aa99-0207e28446e4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:31.008117 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-23761faf-a578-45d6-aa99-0207e28446e4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:31.008582 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-23761faf-a578-45d6-aa99-0207e28446e4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:31.009277 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-23761faf-a578-45d6-aa99-0207e28446e4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:31.009733 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-23761faf-a578-45d6-aa99-0207e28446e4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:31.010191 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-23761faf-a578-45d6-aa99-0207e28446e4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:31.154444 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-23761faf-a578-45d6-aa99-0207e28446e4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:31.163171 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-23761faf-a578-45d6-aa99-0207e28446e4 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1265 microversion: 2.1 time: 0.168104 Oct 24 09:03:31.163954 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 564/1116] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:30 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1265 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:31.616013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5a3a5281-b6e4-4bc4-9dec-3d8c0f9b327c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:31.616732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-5a3a5281-b6e4-4bc4-9dec-3d8c0f9b327c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:31.623595 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5a3a5281-b6e4-4bc4-9dec-3d8c0f9b327c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:31.624394 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5a3a5281-b6e4-4bc4-9dec-3d8c0f9b327c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:31.624943 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5a3a5281-b6e4-4bc4-9dec-3d8c0f9b327c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:31.625654 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5a3a5281-b6e4-4bc4-9dec-3d8c0f9b327c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:31.626292 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5a3a5281-b6e4-4bc4-9dec-3d8c0f9b327c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:31.627081 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5a3a5281-b6e4-4bc4-9dec-3d8c0f9b327c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:31.695934 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5a3a5281-b6e4-4bc4-9dec-3d8c0f9b327c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:31.704709 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5a3a5281-b6e4-4bc4-9dec-3d8c0f9b327c tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1271 microversion: 2.1 time: 0.092447 Oct 24 09:03:31.705490 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 553/1117] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:31 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1271 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:32.187877 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-cee7435b-0016-4378-8584-a8fcaab483e5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:32.188644 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-cee7435b-0016-4378-8584-a8fcaab483e5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:32.195788 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cee7435b-0016-4378-8584-a8fcaab483e5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:32.196386 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cee7435b-0016-4378-8584-a8fcaab483e5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:32.197021 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cee7435b-0016-4378-8584-a8fcaab483e5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:32.197963 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cee7435b-0016-4378-8584-a8fcaab483e5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:32.199066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cee7435b-0016-4378-8584-a8fcaab483e5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:32.200325 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cee7435b-0016-4378-8584-a8fcaab483e5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:32.417585 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-cee7435b-0016-4378-8584-a8fcaab483e5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:32.426934 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-cee7435b-0016-4378-8584-a8fcaab483e5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1265 microversion: 2.1 time: 0.244588 Oct 24 09:03:32.427632 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 565/1118] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:32 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1265 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:32.674323 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-5ee1abc9-2a5d-403f-b920-2ba1c70d8abb req-352c3cb1-35ab-49a6-ada9-b184363cd675 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8b7fd37f-353b-4397-b501-63a1b8155844", "tag": "9b1ea7a2-0ca0-4235-bce7-98d95cd10cf8"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:03:32.679992 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=73,nova_api:UPDATE=2 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:03:32.687674 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-5ee1abc9-2a5d-403f-b920-2ba1c70d8abb req-352c3cb1-35ab-49a6-ada9-b184363cd675 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:32.688271 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-5ee1abc9-2a5d-403f-b920-2ba1c70d8abb req-352c3cb1-35ab-49a6-ada9-b184363cd675 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:32.688772 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-5ee1abc9-2a5d-403f-b920-2ba1c70d8abb req-352c3cb1-35ab-49a6-ada9-b184363cd675 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:32.701107 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-5ee1abc9-2a5d-403f-b920-2ba1c70d8abb req-352c3cb1-35ab-49a6-ada9-b184363cd675 service nova] Creating event network-changed:9b1ea7a2-0ca0-4235-bce7-98d95cd10cf8 for instance 8b7fd37f-353b-4397-b501-63a1b8155844 on hv-cinder-maste Oct 24 09:03:32.705745 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-5ee1abc9-2a5d-403f-b920-2ba1c70d8abb req-352c3cb1-35ab-49a6-ada9-b184363cd675 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035161 Oct 24 09:03:32.706415 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 554/1119] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:03:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 09:03:32.728679 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-5867b143-9bde-4654-9847-3d32f7bd09aa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:32.729330 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-5867b143-9bde-4654-9847-3d32f7bd09aa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:32.735464 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5867b143-9bde-4654-9847-3d32f7bd09aa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:32.735958 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5867b143-9bde-4654-9847-3d32f7bd09aa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:32.736439 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5867b143-9bde-4654-9847-3d32f7bd09aa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:32.737140 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5867b143-9bde-4654-9847-3d32f7bd09aa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:32.737598 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5867b143-9bde-4654-9847-3d32f7bd09aa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:32.738028 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5867b143-9bde-4654-9847-3d32f7bd09aa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:32.950306 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-5867b143-9bde-4654-9847-3d32f7bd09aa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:32.958157 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-5867b143-9bde-4654-9847-3d32f7bd09aa tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1313 microversion: 2.1 time: 0.233222 Oct 24 09:03:32.958658 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 566/1120] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:32 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1313 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:33.451840 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6a604248-8682-433d-a346-94a74de01ded tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:33.452592 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6a604248-8682-433d-a346-94a74de01ded tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:33.458892 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a604248-8682-433d-a346-94a74de01ded tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:33.459377 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a604248-8682-433d-a346-94a74de01ded tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:33.459814 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a604248-8682-433d-a346-94a74de01ded tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:33.460534 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a604248-8682-433d-a346-94a74de01ded tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:33.460988 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a604248-8682-433d-a346-94a74de01ded tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:33.461408 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6a604248-8682-433d-a346-94a74de01ded tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:33.598124 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6a604248-8682-433d-a346-94a74de01ded tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:33.605732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6a604248-8682-433d-a346-94a74de01ded tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1265 microversion: 2.1 time: 0.159013 Oct 24 09:03:33.606981 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 555/1121] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:33 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1265 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:33.981898 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-fd10e161-669d-4389-9a0d-516d3f9f0563 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:33.982413 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-fd10e161-669d-4389-9a0d-516d3f9f0563 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:33.989069 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fd10e161-669d-4389-9a0d-516d3f9f0563 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:33.989590 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fd10e161-669d-4389-9a0d-516d3f9f0563 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:33.990031 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fd10e161-669d-4389-9a0d-516d3f9f0563 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:33.990770 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fd10e161-669d-4389-9a0d-516d3f9f0563 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:33.991222 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fd10e161-669d-4389-9a0d-516d3f9f0563 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:33.991646 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-fd10e161-669d-4389-9a0d-516d3f9f0563 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:34.121724 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-fd10e161-669d-4389-9a0d-516d3f9f0563 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:34.130094 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-fd10e161-669d-4389-9a0d-516d3f9f0563 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1313 microversion: 2.1 time: 0.152632 Oct 24 09:03:34.130664 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 567/1122] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:33 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1313 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:34.629797 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e7641597-6032-49db-8bda-fc52701127a3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:34.630413 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e7641597-6032-49db-8bda-fc52701127a3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:34.638664 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e7641597-6032-49db-8bda-fc52701127a3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:34.639197 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e7641597-6032-49db-8bda-fc52701127a3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:34.639636 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e7641597-6032-49db-8bda-fc52701127a3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:34.640391 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e7641597-6032-49db-8bda-fc52701127a3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:34.640840 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e7641597-6032-49db-8bda-fc52701127a3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:34.641259 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e7641597-6032-49db-8bda-fc52701127a3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:34.775606 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e7641597-6032-49db-8bda-fc52701127a3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:34.783710 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e7641597-6032-49db-8bda-fc52701127a3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1265 microversion: 2.1 time: 0.157893 Oct 24 09:03:34.784315 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 556/1123] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:34 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1265 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:35.153489 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8883666d-c59c-4d82-8b18-1d8e3719614a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:35.154015 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8883666d-c59c-4d82-8b18-1d8e3719614a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:35.160035 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8883666d-c59c-4d82-8b18-1d8e3719614a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:35.160586 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8883666d-c59c-4d82-8b18-1d8e3719614a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:35.161037 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8883666d-c59c-4d82-8b18-1d8e3719614a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:35.161743 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8883666d-c59c-4d82-8b18-1d8e3719614a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:35.162229 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8883666d-c59c-4d82-8b18-1d8e3719614a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:35.162658 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8883666d-c59c-4d82-8b18-1d8e3719614a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:35.383612 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-8883666d-c59c-4d82-8b18-1d8e3719614a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:35.392852 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8883666d-c59c-4d82-8b18-1d8e3719614a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1313 microversion: 2.1 time: 0.244266 Oct 24 09:03:35.393436 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 568/1124] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:35 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1313 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:35.806608 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5f7ef30f-513c-447a-8c6f-75da41350e54 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:35.807258 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-5f7ef30f-513c-447a-8c6f-75da41350e54 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:35.814594 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5f7ef30f-513c-447a-8c6f-75da41350e54 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:35.815073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5f7ef30f-513c-447a-8c6f-75da41350e54 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:35.815504 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5f7ef30f-513c-447a-8c6f-75da41350e54 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:35.816225 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5f7ef30f-513c-447a-8c6f-75da41350e54 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:35.816676 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5f7ef30f-513c-447a-8c6f-75da41350e54 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:35.817090 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5f7ef30f-513c-447a-8c6f-75da41350e54 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:35.950207 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5f7ef30f-513c-447a-8c6f-75da41350e54 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:35.957833 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5f7ef30f-513c-447a-8c6f-75da41350e54 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1265 microversion: 2.1 time: 0.155082 Oct 24 09:03:35.958401 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 557/1125] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:35 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1265 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:36.416516 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-df4ba9a9-22ff-49a5-b634-f7ebbd1b0359 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:36.417133 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-df4ba9a9-22ff-49a5-b634-f7ebbd1b0359 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:36.424114 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-df4ba9a9-22ff-49a5-b634-f7ebbd1b0359 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:36.424632 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-df4ba9a9-22ff-49a5-b634-f7ebbd1b0359 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:36.425069 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-df4ba9a9-22ff-49a5-b634-f7ebbd1b0359 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:36.425771 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-df4ba9a9-22ff-49a5-b634-f7ebbd1b0359 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:36.426223 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-df4ba9a9-22ff-49a5-b634-f7ebbd1b0359 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:36.426653 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-df4ba9a9-22ff-49a5-b634-f7ebbd1b0359 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:36.603748 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-df4ba9a9-22ff-49a5-b634-f7ebbd1b0359 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:36.612828 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-df4ba9a9-22ff-49a5-b634-f7ebbd1b0359 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1313 microversion: 2.1 time: 0.200150 Oct 24 09:03:36.613409 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 569/1126] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:36 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1313 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:36.985551 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f10b14a5-0c2b-411e-8176-ea8619159d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:36.986031 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f10b14a5-0c2b-411e-8176-ea8619159d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:36.992715 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f10b14a5-0c2b-411e-8176-ea8619159d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:36.993566 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f10b14a5-0c2b-411e-8176-ea8619159d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:36.994999 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f10b14a5-0c2b-411e-8176-ea8619159d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:36.996012 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f10b14a5-0c2b-411e-8176-ea8619159d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:36.997084 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f10b14a5-0c2b-411e-8176-ea8619159d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:36.997660 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f10b14a5-0c2b-411e-8176-ea8619159d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:37.156078 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f10b14a5-0c2b-411e-8176-ea8619159d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:37.166459 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f10b14a5-0c2b-411e-8176-ea8619159d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1265 microversion: 2.1 time: 0.189226 Oct 24 09:03:37.176442 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 558/1127] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:36 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1265 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:37.636796 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-3ab13b47-32a8-436b-b279-056a3c4712a1 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:37.637601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-3ab13b47-32a8-436b-b279-056a3c4712a1 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:37.643952 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ab13b47-32a8-436b-b279-056a3c4712a1 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:37.644463 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ab13b47-32a8-436b-b279-056a3c4712a1 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:37.644910 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ab13b47-32a8-436b-b279-056a3c4712a1 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:37.645604 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ab13b47-32a8-436b-b279-056a3c4712a1 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:37.646056 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ab13b47-32a8-436b-b279-056a3c4712a1 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:37.646480 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3ab13b47-32a8-436b-b279-056a3c4712a1 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:37.780606 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-3ab13b47-32a8-436b-b279-056a3c4712a1 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:37.788607 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-3ab13b47-32a8-436b-b279-056a3c4712a1 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1313 microversion: 2.1 time: 0.155658 Oct 24 09:03:37.789483 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 570/1128] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:37 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1313 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:38.190741 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-9ba8fbbc-4766-4794-b2ac-9a2cab655491 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:38.191357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-9ba8fbbc-4766-4794-b2ac-9a2cab655491 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:38.197788 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9ba8fbbc-4766-4794-b2ac-9a2cab655491 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:38.198278 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9ba8fbbc-4766-4794-b2ac-9a2cab655491 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:38.198715 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9ba8fbbc-4766-4794-b2ac-9a2cab655491 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:38.200040 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9ba8fbbc-4766-4794-b2ac-9a2cab655491 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:38.200663 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9ba8fbbc-4766-4794-b2ac-9a2cab655491 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:38.201097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-9ba8fbbc-4766-4794-b2ac-9a2cab655491 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:38.343500 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-9ba8fbbc-4766-4794-b2ac-9a2cab655491 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:38.352666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-9ba8fbbc-4766-4794-b2ac-9a2cab655491 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1265 microversion: 2.1 time: 0.165547 Oct 24 09:03:38.353268 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 559/1129] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:38 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1265 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:38.811326 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ab0db9cd-cf7a-4a8f-ac60-ff2c44a0cfbb tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:38.811937 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ab0db9cd-cf7a-4a8f-ac60-ff2c44a0cfbb tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:38.818301 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ab0db9cd-cf7a-4a8f-ac60-ff2c44a0cfbb tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:38.818783 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ab0db9cd-cf7a-4a8f-ac60-ff2c44a0cfbb tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:38.819217 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ab0db9cd-cf7a-4a8f-ac60-ff2c44a0cfbb tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:38.819906 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ab0db9cd-cf7a-4a8f-ac60-ff2c44a0cfbb tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:38.820393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ab0db9cd-cf7a-4a8f-ac60-ff2c44a0cfbb tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:38.820814 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ab0db9cd-cf7a-4a8f-ac60-ff2c44a0cfbb tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:38.961596 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ab0db9cd-cf7a-4a8f-ac60-ff2c44a0cfbb tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:38.969439 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ab0db9cd-cf7a-4a8f-ac60-ff2c44a0cfbb tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1313 microversion: 2.1 time: 0.162521 Oct 24 09:03:38.970010 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 571/1130] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:38 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1313 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:39.377373 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-7bc256e1-c175-44a9-ad68-b5b0e46a80b2 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:39.377990 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-7bc256e1-c175-44a9-ad68-b5b0e46a80b2 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:39.384444 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7bc256e1-c175-44a9-ad68-b5b0e46a80b2 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:39.384928 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7bc256e1-c175-44a9-ad68-b5b0e46a80b2 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:39.385371 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7bc256e1-c175-44a9-ad68-b5b0e46a80b2 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:39.386074 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7bc256e1-c175-44a9-ad68-b5b0e46a80b2 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:39.386532 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7bc256e1-c175-44a9-ad68-b5b0e46a80b2 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:39.387579 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-7bc256e1-c175-44a9-ad68-b5b0e46a80b2 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:39.480659 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=4 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:03:39.514391 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-7bc256e1-c175-44a9-ad68-b5b0e46a80b2 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:39.521877 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-7bc256e1-c175-44a9-ad68-b5b0e46a80b2 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1265 microversion: 2.1 time: 0.148375 Oct 24 09:03:39.522440 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 560/1131] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:39 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1265 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:39.993823 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-982441b4-e3d6-41b3-8523-57d6eff89f87 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:39.994733 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-982441b4-e3d6-41b3-8523-57d6eff89f87 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:39.997174 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=4 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:03:39.997841 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=12 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:03:40.004699 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-982441b4-e3d6-41b3-8523-57d6eff89f87 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:40.005221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-982441b4-e3d6-41b3-8523-57d6eff89f87 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:40.005702 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-982441b4-e3d6-41b3-8523-57d6eff89f87 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:40.006469 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-982441b4-e3d6-41b3-8523-57d6eff89f87 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:40.006961 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-982441b4-e3d6-41b3-8523-57d6eff89f87 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:40.007451 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-982441b4-e3d6-41b3-8523-57d6eff89f87 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:40.141976 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-982441b4-e3d6-41b3-8523-57d6eff89f87 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:40.149549 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-982441b4-e3d6-41b3-8523-57d6eff89f87 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1313 microversion: 2.1 time: 0.159766 Oct 24 09:03:40.150108 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 572/1132] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:39 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1313 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 09:03:40.545546 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-5756d948-c4eb-41bb-a156-9cf7a9843dc6 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:40.546166 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-5756d948-c4eb-41bb-a156-9cf7a9843dc6 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:40.552704 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5756d948-c4eb-41bb-a156-9cf7a9843dc6 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:40.553194 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5756d948-c4eb-41bb-a156-9cf7a9843dc6 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:40.553637 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5756d948-c4eb-41bb-a156-9cf7a9843dc6 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:40.554362 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5756d948-c4eb-41bb-a156-9cf7a9843dc6 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:40.554817 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5756d948-c4eb-41bb-a156-9cf7a9843dc6 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:40.555244 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-5756d948-c4eb-41bb-a156-9cf7a9843dc6 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:40.684558 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-5756d948-c4eb-41bb-a156-9cf7a9843dc6 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:40.692150 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-5756d948-c4eb-41bb-a156-9cf7a9843dc6 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1265 microversion: 2.1 time: 0.150664 Oct 24 09:03:40.692715 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 561/1133] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:40 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1265 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 09:03:41.172251 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-71097460-2af6-4959-aac0-c67ff05e8da7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:41.172876 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-71097460-2af6-4959-aac0-c67ff05e8da7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:41.179262 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-71097460-2af6-4959-aac0-c67ff05e8da7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:41.179752 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-71097460-2af6-4959-aac0-c67ff05e8da7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:41.180221 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-71097460-2af6-4959-aac0-c67ff05e8da7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:41.180924 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-71097460-2af6-4959-aac0-c67ff05e8da7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:41.181381 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-71097460-2af6-4959-aac0-c67ff05e8da7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:41.181808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-71097460-2af6-4959-aac0-c67ff05e8da7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:41.311977 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-71097460-2af6-4959-aac0-c67ff05e8da7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:41.319698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-71097460-2af6-4959-aac0-c67ff05e8da7 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1313 microversion: 2.1 time: 0.151465 Oct 24 09:03:41.320304 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 573/1134] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:41 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1313 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:41.424616 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-e0ab6767-41e1-4a5b-9217-4de02f32f3ad req-c62d964d-b588-424e-9eb2-4323230b8e11 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1812c1d4-2552-4a5e-9e55-42c7eec001cd", "name": "network-vif-plugged", "status": "completed", "tag": "f4ba7cc3-9768-4fde-9952-2e905e5b30e5"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:03:41.434825 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-e0ab6767-41e1-4a5b-9217-4de02f32f3ad req-c62d964d-b588-424e-9eb2-4323230b8e11 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:41.435423 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-e0ab6767-41e1-4a5b-9217-4de02f32f3ad req-c62d964d-b588-424e-9eb2-4323230b8e11 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:41.435847 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-e0ab6767-41e1-4a5b-9217-4de02f32f3ad req-c62d964d-b588-424e-9eb2-4323230b8e11 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:41.446818 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-e0ab6767-41e1-4a5b-9217-4de02f32f3ad req-c62d964d-b588-424e-9eb2-4323230b8e11 service nova] Creating event network-vif-plugged:f4ba7cc3-9768-4fde-9952-2e905e5b30e5 for instance 1812c1d4-2552-4a5e-9e55-42c7eec001cd on hv-cinder-maste Oct 24 09:03:41.451878 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-e0ab6767-41e1-4a5b-9217-4de02f32f3ad req-c62d964d-b588-424e-9eb2-4323230b8e11 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031028 Oct 24 09:03:41.452514 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 562/1135] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:03:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:03:41.714972 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b951dff8-efd4-4024-8203-810b1952a2f7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:41.715955 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b951dff8-efd4-4024-8203-810b1952a2f7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:41.721867 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b951dff8-efd4-4024-8203-810b1952a2f7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:41.722400 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b951dff8-efd4-4024-8203-810b1952a2f7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:41.722843 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b951dff8-efd4-4024-8203-810b1952a2f7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:41.723553 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b951dff8-efd4-4024-8203-810b1952a2f7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:41.724016 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b951dff8-efd4-4024-8203-810b1952a2f7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:41.724481 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b951dff8-efd4-4024-8203-810b1952a2f7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:41.862291 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-b951dff8-efd4-4024-8203-810b1952a2f7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:41.870161 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b951dff8-efd4-4024-8203-810b1952a2f7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1265 microversion: 2.1 time: 0.159223 Oct 24 09:03:41.870724 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 574/1136] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:41 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1265 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:42.344575 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-fbf25165-9f41-4151-9ffc-cfd4aeb15809 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:42.345186 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-fbf25165-9f41-4151-9ffc-cfd4aeb15809 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:42.351292 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fbf25165-9f41-4151-9ffc-cfd4aeb15809 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:42.351764 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fbf25165-9f41-4151-9ffc-cfd4aeb15809 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:42.352290 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fbf25165-9f41-4151-9ffc-cfd4aeb15809 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:42.352975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fbf25165-9f41-4151-9ffc-cfd4aeb15809 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:42.353420 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fbf25165-9f41-4151-9ffc-cfd4aeb15809 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:42.353836 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fbf25165-9f41-4151-9ffc-cfd4aeb15809 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:42.372425 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:03:42.486196 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fbf25165-9f41-4151-9ffc-cfd4aeb15809 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:42.493820 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-fbf25165-9f41-4151-9ffc-cfd4aeb15809 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1313 microversion: 2.1 time: 0.154658 Oct 24 09:03:42.494649 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 563/1137] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:42 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1313 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:42.893691 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b6f2da71-d7dd-42e5-91ba-d77830282300 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:42.894284 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b6f2da71-d7dd-42e5-91ba-d77830282300 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:42.900734 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6f2da71-d7dd-42e5-91ba-d77830282300 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:42.901226 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6f2da71-d7dd-42e5-91ba-d77830282300 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:42.901651 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6f2da71-d7dd-42e5-91ba-d77830282300 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:42.902337 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6f2da71-d7dd-42e5-91ba-d77830282300 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:42.903956 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6f2da71-d7dd-42e5-91ba-d77830282300 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:42.904497 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b6f2da71-d7dd-42e5-91ba-d77830282300 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:43.033716 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-b6f2da71-d7dd-42e5-91ba-d77830282300 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:43.041302 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b6f2da71-d7dd-42e5-91ba-d77830282300 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1265 microversion: 2.1 time: 0.151478 Oct 24 09:03:43.041867 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 575/1138] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:42 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1265 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:43.472418 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-9294beb1-0771-4a4c-815c-ef3930983e29 req-c40d05b4-2ff1-4b77-9add-e22567777625 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8b7fd37f-353b-4397-b501-63a1b8155844", "name": "network-vif-plugged", "status": "completed", "tag": "9b1ea7a2-0ca0-4235-bce7-98d95cd10cf8"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:03:43.482749 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-9294beb1-0771-4a4c-815c-ef3930983e29 req-c40d05b4-2ff1-4b77-9add-e22567777625 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:43.483338 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-9294beb1-0771-4a4c-815c-ef3930983e29 req-c40d05b4-2ff1-4b77-9add-e22567777625 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:43.483693 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-9294beb1-0771-4a4c-815c-ef3930983e29 req-c40d05b4-2ff1-4b77-9add-e22567777625 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:43.495917 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-9294beb1-0771-4a4c-815c-ef3930983e29 req-c40d05b4-2ff1-4b77-9add-e22567777625 service nova] Creating event network-vif-plugged:9b1ea7a2-0ca0-4235-bce7-98d95cd10cf8 for instance 8b7fd37f-353b-4397-b501-63a1b8155844 on hv-cinder-maste Oct 24 09:03:43.500791 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-9294beb1-0771-4a4c-815c-ef3930983e29 req-c40d05b4-2ff1-4b77-9add-e22567777625 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032277 Oct 24 09:03:43.501375 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 564/1139] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:03:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:03:43.517935 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f1822bd4-28d1-4101-b7dd-b7f3fb71ed4f tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:43.518555 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f1822bd4-28d1-4101-b7dd-b7f3fb71ed4f tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:43.524446 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f1822bd4-28d1-4101-b7dd-b7f3fb71ed4f tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:43.524927 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f1822bd4-28d1-4101-b7dd-b7f3fb71ed4f tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:43.525364 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f1822bd4-28d1-4101-b7dd-b7f3fb71ed4f tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:43.526089 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f1822bd4-28d1-4101-b7dd-b7f3fb71ed4f tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:43.526540 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f1822bd4-28d1-4101-b7dd-b7f3fb71ed4f tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:43.526965 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f1822bd4-28d1-4101-b7dd-b7f3fb71ed4f tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:43.664624 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f1822bd4-28d1-4101-b7dd-b7f3fb71ed4f tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:43.672852 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f1822bd4-28d1-4101-b7dd-b7f3fb71ed4f tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1313 microversion: 2.1 time: 0.158726 Oct 24 09:03:43.673446 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 576/1140] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:43 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1313 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:44.065201 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a10ea869-0f17-4eb7-a27c-857eff0ee61c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:44.065907 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-a10ea869-0f17-4eb7-a27c-857eff0ee61c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:44.071981 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a10ea869-0f17-4eb7-a27c-857eff0ee61c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:44.072495 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a10ea869-0f17-4eb7-a27c-857eff0ee61c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:44.072933 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a10ea869-0f17-4eb7-a27c-857eff0ee61c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:44.073618 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a10ea869-0f17-4eb7-a27c-857eff0ee61c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:44.074070 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a10ea869-0f17-4eb7-a27c-857eff0ee61c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:44.074693 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a10ea869-0f17-4eb7-a27c-857eff0ee61c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:44.207463 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-a10ea869-0f17-4eb7-a27c-857eff0ee61c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:44.215794 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-a10ea869-0f17-4eb7-a27c-857eff0ee61c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1452 microversion: 2.1 time: 0.154077 Oct 24 09:03:44.216411 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 565/1141] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:44 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1452 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:44.237677 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-75cd787a-e40b-482b-9205-d024c3dc6f12 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:44.238297 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-75cd787a-e40b-482b-9205-d024c3dc6f12 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:44.245039 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-75cd787a-e40b-482b-9205-d024c3dc6f12 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:44.245528 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-75cd787a-e40b-482b-9205-d024c3dc6f12 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:44.245970 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-75cd787a-e40b-482b-9205-d024c3dc6f12 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:44.246975 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-75cd787a-e40b-482b-9205-d024c3dc6f12 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:44.247294 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-75cd787a-e40b-482b-9205-d024c3dc6f12 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:44.247724 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-75cd787a-e40b-482b-9205-d024c3dc6f12 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:44.379156 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-75cd787a-e40b-482b-9205-d024c3dc6f12 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:44.386779 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-75cd787a-e40b-482b-9205-d024c3dc6f12 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1452 microversion: 2.1 time: 0.153012 Oct 24 09:03:44.387890 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 577/1142] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:44 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1452 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:44.695668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-aaa3a645-717f-4a55-a6f8-f4e73f652ad6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:44.696331 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-aaa3a645-717f-4a55-a6f8-f4e73f652ad6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:44.703309 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aaa3a645-717f-4a55-a6f8-f4e73f652ad6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:44.703797 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aaa3a645-717f-4a55-a6f8-f4e73f652ad6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:44.704268 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aaa3a645-717f-4a55-a6f8-f4e73f652ad6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:44.704976 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aaa3a645-717f-4a55-a6f8-f4e73f652ad6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:44.705438 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aaa3a645-717f-4a55-a6f8-f4e73f652ad6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:44.705865 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-aaa3a645-717f-4a55-a6f8-f4e73f652ad6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:44.899384 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-aaa3a645-717f-4a55-a6f8-f4e73f652ad6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:44.907025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-aaa3a645-717f-4a55-a6f8-f4e73f652ad6 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1313 microversion: 2.1 time: 0.215229 Oct 24 09:03:44.907599 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 566/1143] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:44 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1313 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:45.930599 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-5e4c25b1-d94f-4ca6-9683-16761da48e0d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:45.931250 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-5e4c25b1-d94f-4ca6-9683-16761da48e0d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:45.938071 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e4c25b1-d94f-4ca6-9683-16761da48e0d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:45.938560 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e4c25b1-d94f-4ca6-9683-16761da48e0d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:45.939005 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e4c25b1-d94f-4ca6-9683-16761da48e0d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:45.939708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e4c25b1-d94f-4ca6-9683-16761da48e0d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:45.940192 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e4c25b1-d94f-4ca6-9683-16761da48e0d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:45.940625 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-5e4c25b1-d94f-4ca6-9683-16761da48e0d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:46.177815 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-5e4c25b1-d94f-4ca6-9683-16761da48e0d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:46.185608 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-5e4c25b1-d94f-4ca6-9683-16761da48e0d tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1498 microversion: 2.1 time: 0.258740 Oct 24 09:03:46.185973 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 578/1144] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:45 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1498 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:46.367298 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-f9ea638a-e63d-447f-a69a-7f3588d60332 req-47f877a4-1e58-440c-863f-91a4f6f5d780 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1812c1d4-2552-4a5e-9e55-42c7eec001cd", "tag": "f4ba7cc3-9768-4fde-9952-2e905e5b30e5"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:03:46.378795 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-f9ea638a-e63d-447f-a69a-7f3588d60332 req-47f877a4-1e58-440c-863f-91a4f6f5d780 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:46.379433 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-f9ea638a-e63d-447f-a69a-7f3588d60332 req-47f877a4-1e58-440c-863f-91a4f6f5d780 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:46.380162 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-f9ea638a-e63d-447f-a69a-7f3588d60332 req-47f877a4-1e58-440c-863f-91a4f6f5d780 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:46.392076 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-f9ea638a-e63d-447f-a69a-7f3588d60332 req-47f877a4-1e58-440c-863f-91a4f6f5d780 service nova] Creating event network-changed:f4ba7cc3-9768-4fde-9952-2e905e5b30e5 for instance 1812c1d4-2552-4a5e-9e55-42c7eec001cd on hv-cinder-maste Oct 24 09:03:46.397401 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-f9ea638a-e63d-447f-a69a-7f3588d60332 req-47f877a4-1e58-440c-863f-91a4f6f5d780 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033904 Oct 24 09:03:46.398477 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 567/1145] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:03:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:03:46.748761 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a355c8d7-00e2-45ea-93fb-4f13e5e51168 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:46.749422 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a355c8d7-00e2-45ea-93fb-4f13e5e51168 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:46.755632 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a355c8d7-00e2-45ea-93fb-4f13e5e51168 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:46.756339 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a355c8d7-00e2-45ea-93fb-4f13e5e51168 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:46.756787 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a355c8d7-00e2-45ea-93fb-4f13e5e51168 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:46.757490 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a355c8d7-00e2-45ea-93fb-4f13e5e51168 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:46.758134 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a355c8d7-00e2-45ea-93fb-4f13e5e51168 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:46.758723 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a355c8d7-00e2-45ea-93fb-4f13e5e51168 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:46.784918 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a355c8d7-00e2-45ea-93fb-4f13e5e51168 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Going to try to terminate instance {{(pid=130395) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 09:03:46.799329 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=100 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:03:46.865489 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a355c8d7-00e2-45ea-93fb-4f13e5e51168 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "DELETE /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 204 len: 0 microversion: 2.1 time: 0.121220 Oct 24 09:03:46.866564 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 579/1146] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 09:03:46 2022] DELETE /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 09:03:46.876911 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-acd6fc12-37e0-40c1-9c1f-490e90833090 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:46.877814 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-acd6fc12-37e0-40c1-9c1f-490e90833090 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:46.884181 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-acd6fc12-37e0-40c1-9c1f-490e90833090 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:46.884675 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-acd6fc12-37e0-40c1-9c1f-490e90833090 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:46.885149 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-acd6fc12-37e0-40c1-9c1f-490e90833090 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:46.885856 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-acd6fc12-37e0-40c1-9c1f-490e90833090 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:46.886310 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-acd6fc12-37e0-40c1-9c1f-490e90833090 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:46.886732 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-acd6fc12-37e0-40c1-9c1f-490e90833090 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:47.019944 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-acd6fc12-37e0-40c1-9c1f-490e90833090 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:47.031251 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-acd6fc12-37e0-40c1-9c1f-490e90833090 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1504 microversion: 2.1 time: 0.159998 Oct 24 09:03:47.032010 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 568/1147] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:46 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1504 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 09:03:48.055905 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b37e51f0-395b-49b2-951e-a2b9e3bef37a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:48.056457 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b37e51f0-395b-49b2-951e-a2b9e3bef37a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:48.063117 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b37e51f0-395b-49b2-951e-a2b9e3bef37a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:48.063598 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b37e51f0-395b-49b2-951e-a2b9e3bef37a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:48.064047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b37e51f0-395b-49b2-951e-a2b9e3bef37a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:48.064753 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b37e51f0-395b-49b2-951e-a2b9e3bef37a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:48.065204 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b37e51f0-395b-49b2-951e-a2b9e3bef37a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:48.065643 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b37e51f0-395b-49b2-951e-a2b9e3bef37a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:48.202597 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-b37e51f0-395b-49b2-951e-a2b9e3bef37a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:48.210357 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b37e51f0-395b-49b2-951e-a2b9e3bef37a tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1504 microversion: 2.1 time: 0.159096 Oct 24 09:03:48.210922 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 580/1148] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:48 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1504 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:03:49.015486 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-a607f2b0-ac8d-4dc0-8072-7d3b1c081cb7 req-7de522db-5f5d-4de3-b555-fd124578c538 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8b7fd37f-353b-4397-b501-63a1b8155844", "name": "network-vif-deleted", "tag": "9b1ea7a2-0ca0-4235-bce7-98d95cd10cf8"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:03:49.026407 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-a607f2b0-ac8d-4dc0-8072-7d3b1c081cb7 req-7de522db-5f5d-4de3-b555-fd124578c538 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:49.027159 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-a607f2b0-ac8d-4dc0-8072-7d3b1c081cb7 req-7de522db-5f5d-4de3-b555-fd124578c538 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:49.027625 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-a607f2b0-ac8d-4dc0-8072-7d3b1c081cb7 req-7de522db-5f5d-4de3-b555-fd124578c538 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:49.039464 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-a607f2b0-ac8d-4dc0-8072-7d3b1c081cb7 req-7de522db-5f5d-4de3-b555-fd124578c538 service nova] Creating event network-vif-deleted:9b1ea7a2-0ca0-4235-bce7-98d95cd10cf8 for instance 8b7fd37f-353b-4397-b501-63a1b8155844 on hv-cinder-maste Oct 24 09:03:49.045222 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-a607f2b0-ac8d-4dc0-8072-7d3b1c081cb7 req-7de522db-5f5d-4de3-b555-fd124578c538 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034375 Oct 24 09:03:49.045816 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 569/1149] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:03:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:03:49.235066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f0f60e17-9d28-4b64-974d-87d5ced67810 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:49.235682 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f0f60e17-9d28-4b64-974d-87d5ced67810 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:49.242097 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f0f60e17-9d28-4b64-974d-87d5ced67810 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:49.242583 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f0f60e17-9d28-4b64-974d-87d5ced67810 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:49.243021 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f0f60e17-9d28-4b64-974d-87d5ced67810 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:49.243746 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f0f60e17-9d28-4b64-974d-87d5ced67810 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:49.244960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f0f60e17-9d28-4b64-974d-87d5ced67810 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:49.245386 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f0f60e17-9d28-4b64-974d-87d5ced67810 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:49.276743 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f0f60e17-9d28-4b64-974d-87d5ced67810 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:49.277234 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f0f60e17-9d28-4b64-974d-87d5ced67810 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:49.278187 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f0f60e17-9d28-4b64-974d-87d5ced67810 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:49.278636 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.objects.instance [None req-f0f60e17-9d28-4b64-974d-87d5ced67810 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lazy-loading 'fault' on Instance uuid 8b7fd37f-353b-4397-b501-63a1b8155844 {{(pid=130395) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 09:03:49.320499 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f0f60e17-9d28-4b64-974d-87d5ced67810 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f452f27fc0d34f73aa22da2a09d54eb8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ccb79814f5432cb21c619e974fecd8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:03:49.328188 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f0f60e17-9d28-4b64-974d-87d5ced67810 tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 200 len: 1303 microversion: 2.1 time: 0.097658 Oct 24 09:03:49.328758 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 581/1150] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:49 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 1303 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 09:03:50.352594 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b4841f28-9ff8-41c1-8911-4dc1968f77dc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:03:50.353243 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b4841f28-9ff8-41c1-8911-4dc1968f77dc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] [instance: 8b7fd37f-353b-4397-b501-63a1b8155844] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:03:50.360491 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b4841f28-9ff8-41c1-8911-4dc1968f77dc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:50.360975 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b4841f28-9ff8-41c1-8911-4dc1968f77dc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:50.361409 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b4841f28-9ff8-41c1-8911-4dc1968f77dc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:50.362101 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b4841f28-9ff8-41c1-8911-4dc1968f77dc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:03:50.362557 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b4841f28-9ff8-41c1-8911-4dc1968f77dc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:03:50.362981 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b4841f28-9ff8-41c1-8911-4dc1968f77dc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:03:50.376796 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-b4841f28-9ff8-41c1-8911-4dc1968f77dc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] HTTP exception thrown: Instance 8b7fd37f-353b-4397-b501-63a1b8155844 could not be found. Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b4841f28-9ff8-41c1-8911-4dc1968f77dc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Request method failure captured: Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 HTTP/1.1 Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlS_ru2bHllLLO725iFN5fiACg8PQIsNVjiesAdzAyp0HjdBT5pVV_4eVQ3005xtqUBAbB3XSMIABOqBsnXCCzrEMcbPGiQnhbGz2k_vYJDZeOv0GDVi93VJ7Ne3BGBQCU9kRzxSKjgavwF0nTNzLX5CrZwV-_qnkZMIhgXflopg3h0 Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: d0ccb79814f5432cb21c619e974fecd8 Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-VolumesActionsTest-1529311557 Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 09:03:50.378393 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 09:03:50.380075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/d0ccb79814f5432cb21c619e974fecd8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_d0ccb79814f5432cb21c619e974fecd8", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/d0ccb79814f5432cb21c619e974fecd8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/d0ccb79814f5432cb21c619e974fecd8"}]}] Oct 24 09:03:50.380075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-VolumesActionsTest-1529311557 Oct 24 09:03:50.380075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: d0ccb79814f5432cb21c619e974fecd8 Oct 24 09:03:50.380075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-VolumesActionsTest-1529311557 Oct 24 09:03:50.380075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-VolumesActionsTest-1529311557-project Oct 24 09:03:50.380075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 09:03:50.380075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 09:03:50.380075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: f452f27fc0d34f73aa22da2a09d54eb8 Oct 24 09:03:50.380075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-VolumesActionsTest-1529311557-project Oct 24 09:03:50.380075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 09:03:50.380075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance 8b7fd37f-353b-4397-b501-63a1b8155844 could not be found. Oct 24 09:03:50.380075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': '8b7fd37f-353b-4397-b501-63a1b8155844'} Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8b7fd37f-353b-4397-b501-63a1b8155844 could not be found. Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:03:50.381059 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8b7fd37f-353b-4397-b501-63a1b8155844 could not be found. Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8b7fd37f-353b-4397-b501-63a1b8155844 could not be found. Oct 24 09:03:50.382670 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:03:50.384324 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b4841f28-9ff8-41c1-8911-4dc1968f77dc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] Returning 404 to user: Instance 8b7fd37f-353b-4397-b501-63a1b8155844 could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:03:50.384324 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b4841f28-9ff8-41c1-8911-4dc1968f77dc tempest-VolumesActionsTest-1529311557 tempest-VolumesActionsTest-1529311557-project] 10.222.0.39 "GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844" status: 404 len: 111 microversion: 2.1 time: 0.031581 Oct 24 09:03:50.384324 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 570/1151] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:03:50 2022] GET /compute/v2.1/servers/8b7fd37f-353b-4397-b501-63a1b8155844 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:04:16.645155 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-19acbe42-9890-4b83-a683-b5595857e7bb tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:04:16.647506 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-19acbe42-9890-4b83-a683-b5595857e7bb tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:04:16.649770 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=42,nova_api:UPDATE=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:04:16.650473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=20,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:04:16.656652 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19acbe42-9890-4b83-a683-b5595857e7bb tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:16.657149 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19acbe42-9890-4b83-a683-b5595857e7bb tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:16.657800 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19acbe42-9890-4b83-a683-b5595857e7bb tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:16.658521 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19acbe42-9890-4b83-a683-b5595857e7bb tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:16.658987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19acbe42-9890-4b83-a683-b5595857e7bb tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:16.659421 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-19acbe42-9890-4b83-a683-b5595857e7bb tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:16.738415 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-19acbe42-9890-4b83-a683-b5595857e7bb tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "POST /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/action" status: 202 len: 0 microversion: 2.1 time: 0.097504 Oct 24 09:04:16.739094 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 582/1152] 10.222.0.39 () {60 vars in 1278 bytes} [Mon Oct 24 09:04:16 2022] POST /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/action => generated 0 bytes in 99 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 09:04:16.748285 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e50bc77f-e8d5-4ec3-ad1b-7677c4887035 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:04:16.749553 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e50bc77f-e8d5-4ec3-ad1b-7677c4887035 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:04:16.751798 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=33 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:04:16.752483 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=28 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:04:16.758348 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e50bc77f-e8d5-4ec3-ad1b-7677c4887035 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:16.758847 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e50bc77f-e8d5-4ec3-ad1b-7677c4887035 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:16.759325 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e50bc77f-e8d5-4ec3-ad1b-7677c4887035 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:16.760046 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e50bc77f-e8d5-4ec3-ad1b-7677c4887035 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:16.760590 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e50bc77f-e8d5-4ec3-ad1b-7677c4887035 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:16.761022 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e50bc77f-e8d5-4ec3-ad1b-7677c4887035 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:16.999622 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e50bc77f-e8d5-4ec3-ad1b-7677c4887035 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:04:17.007917 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e50bc77f-e8d5-4ec3-ad1b-7677c4887035 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1750 microversion: 2.1 time: 0.264615 Oct 24 09:04:17.008622 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 571/1153] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:04:16 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1750 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:04:18.035114 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7e50ad9d-ad30-4dd1-813a-6991f56be8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:04:18.035833 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7e50ad9d-ad30-4dd1-813a-6991f56be8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:04:18.042036 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e50ad9d-ad30-4dd1-813a-6991f56be8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:18.042611 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e50ad9d-ad30-4dd1-813a-6991f56be8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:18.043057 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e50ad9d-ad30-4dd1-813a-6991f56be8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:18.044592 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e50ad9d-ad30-4dd1-813a-6991f56be8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:18.044859 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e50ad9d-ad30-4dd1-813a-6991f56be8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:18.045292 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7e50ad9d-ad30-4dd1-813a-6991f56be8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:18.192622 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7e50ad9d-ad30-4dd1-813a-6991f56be8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:04:18.201424 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7e50ad9d-ad30-4dd1-813a-6991f56be8fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1755 microversion: 2.1 time: 0.171499 Oct 24 09:04:18.201999 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 583/1154] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:04:18 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1755 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:04:19.225822 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-33f859c5-db52-4927-8734-adeb2d76a203 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:04:19.226426 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-33f859c5-db52-4927-8734-adeb2d76a203 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:04:19.233666 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-33f859c5-db52-4927-8734-adeb2d76a203 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:19.234144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-33f859c5-db52-4927-8734-adeb2d76a203 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:19.234575 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-33f859c5-db52-4927-8734-adeb2d76a203 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:19.235269 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-33f859c5-db52-4927-8734-adeb2d76a203 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:19.235713 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-33f859c5-db52-4927-8734-adeb2d76a203 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:19.236147 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-33f859c5-db52-4927-8734-adeb2d76a203 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:19.380240 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-33f859c5-db52-4927-8734-adeb2d76a203 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:04:19.389881 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-33f859c5-db52-4927-8734-adeb2d76a203 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1755 microversion: 2.1 time: 0.168506 Oct 24 09:04:19.390469 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 572/1155] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:04:19 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1755 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:04:20.414085 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ef0aa55a-2b23-4399-8c04-22fe0f77ff71 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:04:20.414714 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ef0aa55a-2b23-4399-8c04-22fe0f77ff71 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:04:20.421501 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef0aa55a-2b23-4399-8c04-22fe0f77ff71 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:20.421993 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef0aa55a-2b23-4399-8c04-22fe0f77ff71 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:20.422440 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef0aa55a-2b23-4399-8c04-22fe0f77ff71 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:20.423143 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef0aa55a-2b23-4399-8c04-22fe0f77ff71 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:20.423595 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef0aa55a-2b23-4399-8c04-22fe0f77ff71 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:20.424047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ef0aa55a-2b23-4399-8c04-22fe0f77ff71 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:20.575863 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-ef0aa55a-2b23-4399-8c04-22fe0f77ff71 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:04:20.584198 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ef0aa55a-2b23-4399-8c04-22fe0f77ff71 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1755 microversion: 2.1 time: 0.174086 Oct 24 09:04:20.584995 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 584/1156] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:04:20 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1755 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:04:21.608842 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-630fb520-72d5-46ac-8f41-a4dd97c51be5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:04:21.609685 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-630fb520-72d5-46ac-8f41-a4dd97c51be5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:04:21.622582 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-630fb520-72d5-46ac-8f41-a4dd97c51be5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:21.623097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-630fb520-72d5-46ac-8f41-a4dd97c51be5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:21.623540 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-630fb520-72d5-46ac-8f41-a4dd97c51be5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:21.624297 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-630fb520-72d5-46ac-8f41-a4dd97c51be5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:21.624750 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-630fb520-72d5-46ac-8f41-a4dd97c51be5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:21.625173 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-630fb520-72d5-46ac-8f41-a4dd97c51be5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:21.766605 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-630fb520-72d5-46ac-8f41-a4dd97c51be5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:04:21.774969 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-630fb520-72d5-46ac-8f41-a4dd97c51be5 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1755 microversion: 2.1 time: 0.171504 Oct 24 09:04:21.775530 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 573/1157] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:04:21 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1755 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:04:22.799587 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7581d55b-ce65-4f5c-ac5d-2ba8a0878bda tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:04:22.800243 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7581d55b-ce65-4f5c-ac5d-2ba8a0878bda tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:04:22.806512 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7581d55b-ce65-4f5c-ac5d-2ba8a0878bda tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:22.806998 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7581d55b-ce65-4f5c-ac5d-2ba8a0878bda tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:22.807438 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7581d55b-ce65-4f5c-ac5d-2ba8a0878bda tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:22.808150 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7581d55b-ce65-4f5c-ac5d-2ba8a0878bda tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:22.808621 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7581d55b-ce65-4f5c-ac5d-2ba8a0878bda tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:22.809203 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7581d55b-ce65-4f5c-ac5d-2ba8a0878bda tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:22.955644 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-7581d55b-ce65-4f5c-ac5d-2ba8a0878bda tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:04:22.964253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7581d55b-ce65-4f5c-ac5d-2ba8a0878bda tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1755 microversion: 2.1 time: 0.168659 Oct 24 09:04:22.964870 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 585/1158] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:04:22 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1755 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:04:23.987988 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6e4020b1-73d2-4f03-ba64-1446b7394364 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:04:23.988543 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6e4020b1-73d2-4f03-ba64-1446b7394364 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:04:23.994921 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6e4020b1-73d2-4f03-ba64-1446b7394364 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:23.995394 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6e4020b1-73d2-4f03-ba64-1446b7394364 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:23.996074 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6e4020b1-73d2-4f03-ba64-1446b7394364 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:23.997323 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6e4020b1-73d2-4f03-ba64-1446b7394364 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:23.997771 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6e4020b1-73d2-4f03-ba64-1446b7394364 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:23.998188 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6e4020b1-73d2-4f03-ba64-1446b7394364 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:24.142311 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6e4020b1-73d2-4f03-ba64-1446b7394364 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:04:24.150676 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6e4020b1-73d2-4f03-ba64-1446b7394364 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1755 microversion: 2.1 time: 0.166575 Oct 24 09:04:24.151242 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 574/1159] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:04:23 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1755 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:04:25.174796 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-85b630c9-2cd6-47bd-b80c-67ee0cb8ab37 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:04:25.175425 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-85b630c9-2cd6-47bd-b80c-67ee0cb8ab37 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:04:25.181757 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-85b630c9-2cd6-47bd-b80c-67ee0cb8ab37 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:25.182252 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-85b630c9-2cd6-47bd-b80c-67ee0cb8ab37 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:25.182700 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-85b630c9-2cd6-47bd-b80c-67ee0cb8ab37 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:25.183404 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-85b630c9-2cd6-47bd-b80c-67ee0cb8ab37 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:25.184384 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-85b630c9-2cd6-47bd-b80c-67ee0cb8ab37 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:25.185178 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-85b630c9-2cd6-47bd-b80c-67ee0cb8ab37 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:25.385871 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-85b630c9-2cd6-47bd-b80c-67ee0cb8ab37 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:04:25.395829 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-85b630c9-2cd6-47bd-b80c-67ee0cb8ab37 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1755 microversion: 2.1 time: 0.224816 Oct 24 09:04:25.396540 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 586/1160] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:04:25 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1755 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:04:26.420489 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-2c58ce20-690a-4414-93ed-be2639717518 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:04:26.421131 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-2c58ce20-690a-4414-93ed-be2639717518 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:04:26.427399 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2c58ce20-690a-4414-93ed-be2639717518 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:26.427880 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2c58ce20-690a-4414-93ed-be2639717518 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:26.428360 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2c58ce20-690a-4414-93ed-be2639717518 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:26.429049 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2c58ce20-690a-4414-93ed-be2639717518 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:26.429496 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2c58ce20-690a-4414-93ed-be2639717518 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:26.429917 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-2c58ce20-690a-4414-93ed-be2639717518 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:26.582137 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-2c58ce20-690a-4414-93ed-be2639717518 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:04:26.591570 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-2c58ce20-690a-4414-93ed-be2639717518 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1755 microversion: 2.1 time: 0.175277 Oct 24 09:04:26.592156 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 575/1161] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:04:26 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1755 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:04:27.615373 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-a6006fca-115f-4ca1-a413-65e33ae346e4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:04:27.616020 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-a6006fca-115f-4ca1-a413-65e33ae346e4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:04:27.623498 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a6006fca-115f-4ca1-a413-65e33ae346e4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:27.623978 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a6006fca-115f-4ca1-a413-65e33ae346e4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:27.624451 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a6006fca-115f-4ca1-a413-65e33ae346e4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:27.625147 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a6006fca-115f-4ca1-a413-65e33ae346e4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:27.625603 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a6006fca-115f-4ca1-a413-65e33ae346e4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:27.626030 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-a6006fca-115f-4ca1-a413-65e33ae346e4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:27.764270 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-a6006fca-115f-4ca1-a413-65e33ae346e4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:04:27.772476 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-a6006fca-115f-4ca1-a413-65e33ae346e4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1755 microversion: 2.1 time: 0.160906 Oct 24 09:04:27.773073 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 587/1162] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:04:27 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1755 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:04:28.795865 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-131da9f0-522d-487c-9224-3280e9b117c8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:04:28.796512 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-131da9f0-522d-487c-9224-3280e9b117c8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:04:28.803045 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-131da9f0-522d-487c-9224-3280e9b117c8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:28.803529 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-131da9f0-522d-487c-9224-3280e9b117c8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:28.803964 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-131da9f0-522d-487c-9224-3280e9b117c8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:28.804693 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-131da9f0-522d-487c-9224-3280e9b117c8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:28.805148 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-131da9f0-522d-487c-9224-3280e9b117c8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:28.805576 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-131da9f0-522d-487c-9224-3280e9b117c8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:28.948743 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-131da9f0-522d-487c-9224-3280e9b117c8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:04:28.957280 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-131da9f0-522d-487c-9224-3280e9b117c8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1755 microversion: 2.1 time: 0.165309 Oct 24 09:04:28.957845 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 576/1163] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:04:28 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1755 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:04:29.981553 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-c90f62a6-8c9f-4aa7-9afe-a89817cdfaa8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:04:29.982093 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-c90f62a6-8c9f-4aa7-9afe-a89817cdfaa8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:04:29.988577 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c90f62a6-8c9f-4aa7-9afe-a89817cdfaa8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:29.989060 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c90f62a6-8c9f-4aa7-9afe-a89817cdfaa8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:29.989496 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c90f62a6-8c9f-4aa7-9afe-a89817cdfaa8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:29.990194 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c90f62a6-8c9f-4aa7-9afe-a89817cdfaa8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:29.990650 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c90f62a6-8c9f-4aa7-9afe-a89817cdfaa8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:29.991072 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-c90f62a6-8c9f-4aa7-9afe-a89817cdfaa8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:30.134681 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-c90f62a6-8c9f-4aa7-9afe-a89817cdfaa8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:04:30.143586 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-c90f62a6-8c9f-4aa7-9afe-a89817cdfaa8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1755 microversion: 2.1 time: 0.166018 Oct 24 09:04:30.144157 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 588/1164] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:04:29 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1755 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:04:31.167331 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-39fcc350-cc31-47db-83f8-4ecddc1fbf89 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:04:31.167948 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-39fcc350-cc31-47db-83f8-4ecddc1fbf89 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:04:31.174421 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-39fcc350-cc31-47db-83f8-4ecddc1fbf89 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:31.174903 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-39fcc350-cc31-47db-83f8-4ecddc1fbf89 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:31.175342 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-39fcc350-cc31-47db-83f8-4ecddc1fbf89 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:31.176039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-39fcc350-cc31-47db-83f8-4ecddc1fbf89 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:31.176517 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-39fcc350-cc31-47db-83f8-4ecddc1fbf89 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:31.176942 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-39fcc350-cc31-47db-83f8-4ecddc1fbf89 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:31.319629 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-39fcc350-cc31-47db-83f8-4ecddc1fbf89 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:04:31.328914 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-39fcc350-cc31-47db-83f8-4ecddc1fbf89 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1755 microversion: 2.1 time: 0.165452 Oct 24 09:04:31.329478 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 577/1165] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:04:31 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1755 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:04:32.352095 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-02d18f11-e971-4a2a-a65e-1ae4eea1f677 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:04:32.352784 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-02d18f11-e971-4a2a-a65e-1ae4eea1f677 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:04:32.359452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02d18f11-e971-4a2a-a65e-1ae4eea1f677 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:32.359938 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02d18f11-e971-4a2a-a65e-1ae4eea1f677 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:32.360415 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02d18f11-e971-4a2a-a65e-1ae4eea1f677 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:32.361122 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02d18f11-e971-4a2a-a65e-1ae4eea1f677 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:32.361579 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02d18f11-e971-4a2a-a65e-1ae4eea1f677 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:32.362038 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-02d18f11-e971-4a2a-a65e-1ae4eea1f677 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:32.504351 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-02d18f11-e971-4a2a-a65e-1ae4eea1f677 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:04:32.513164 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-02d18f11-e971-4a2a-a65e-1ae4eea1f677 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1755 microversion: 2.1 time: 0.164293 Oct 24 09:04:32.513705 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 589/1166] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:04:32 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1755 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:04:33.536534 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-988760fb-ad13-4e29-a428-a86bff348b2f tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:04:33.537140 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-988760fb-ad13-4e29-a428-a86bff348b2f tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:04:33.542907 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-988760fb-ad13-4e29-a428-a86bff348b2f tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:33.543484 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-988760fb-ad13-4e29-a428-a86bff348b2f tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:33.544428 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-988760fb-ad13-4e29-a428-a86bff348b2f tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:33.545115 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-988760fb-ad13-4e29-a428-a86bff348b2f tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:33.545562 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-988760fb-ad13-4e29-a428-a86bff348b2f tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:33.545976 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-988760fb-ad13-4e29-a428-a86bff348b2f tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:33.685575 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-988760fb-ad13-4e29-a428-a86bff348b2f tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:04:33.693715 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-988760fb-ad13-4e29-a428-a86bff348b2f tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1755 microversion: 2.1 time: 0.161282 Oct 24 09:04:33.694310 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 578/1167] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:04:33 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1755 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:04:34.717499 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e8493dba-e39c-41dd-80d9-cbe359506da8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:04:34.718108 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e8493dba-e39c-41dd-80d9-cbe359506da8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:04:34.724064 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e8493dba-e39c-41dd-80d9-cbe359506da8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:34.724579 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e8493dba-e39c-41dd-80d9-cbe359506da8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:34.725023 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e8493dba-e39c-41dd-80d9-cbe359506da8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:34.725709 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e8493dba-e39c-41dd-80d9-cbe359506da8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:04:34.726163 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e8493dba-e39c-41dd-80d9-cbe359506da8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:04:34.726586 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e8493dba-e39c-41dd-80d9-cbe359506da8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:04:34.870722 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e8493dba-e39c-41dd-80d9-cbe359506da8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:04:34.879318 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e8493dba-e39c-41dd-80d9-cbe359506da8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1758 microversion: 2.1 time: 0.166169 Oct 24 09:04:34.879880 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 590/1168] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:04:34 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1758 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:05:39.684696 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=16 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:05:39.685516 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=48 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:05:39.686927 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 24 09:05:39.687180 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Mon Oct 24 09:05:39 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/action (ip 10.222.0.39) !!! Oct 24 09:05:39.688797 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 24 09:05:39.688797 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Mon Oct 24 09:05:39 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/action (ip 10.222.0.39) !!! Oct 24 09:05:39.693247 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-3af5b896-bb83-4d31-839b-51a8cb0e6c2d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-188351888"}} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:05:39.693853 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-3af5b896-bb83-4d31-839b-51a8cb0e6c2d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:39.707025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3af5b896-bb83-4d31-839b-51a8cb0e6c2d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:39.707513 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3af5b896-bb83-4d31-839b-51a8cb0e6c2d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:39.707956 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3af5b896-bb83-4d31-839b-51a8cb0e6c2d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:39.708700 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3af5b896-bb83-4d31-839b-51a8cb0e6c2d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:39.709154 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3af5b896-bb83-4d31-839b-51a8cb0e6c2d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:39.709577 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3af5b896-bb83-4d31-839b-51a8cb0e6c2d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:39.845117 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.network.security_group_api [None req-3af5b896-bb83-4d31-839b-51a8cb0e6c2d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Removing security group 84041028-15de-4e5b-bef7-4e848bffde5d from port b8947813-7d62-4e04-8aa8-8ba174d01def Oct 24 09:05:40.424790 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-3af5b896-bb83-4d31-839b-51a8cb0e6c2d tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "POST /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/action" status: 202 len: 0 microversion: 2.1 time: 0.741903 Oct 24 09:05:40.425402 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 579/1169] 10.222.0.39 () {60 vars in 1278 bytes} [Mon Oct 24 09:05:39 2022] POST /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/action => generated 0 bytes in 743 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 09:05:40.431652 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=18 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:05:40.432510 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=58,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:05:40.435373 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 24 09:05:40.435537 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Mon Oct 24 09:05:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.39) !!! Oct 24 09:05:40.440852 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-33ffceec-2b2f-4207-a9cb-150467a837af req-0d7f7066-203b-4916-92d2-175f34743800 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a7260920-be84-4a74-a18e-5b47d53e3db1", "tag": "b8947813-7d62-4e04-8aa8-8ba174d01def"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:05:40.455549 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-33ffceec-2b2f-4207-a9cb-150467a837af req-0d7f7066-203b-4916-92d2-175f34743800 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:40.457099 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-33ffceec-2b2f-4207-a9cb-150467a837af req-0d7f7066-203b-4916-92d2-175f34743800 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:40.457612 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-33ffceec-2b2f-4207-a9cb-150467a837af req-0d7f7066-203b-4916-92d2-175f34743800 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:40.470173 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-33ffceec-2b2f-4207-a9cb-150467a837af req-0d7f7066-203b-4916-92d2-175f34743800 service nova] Creating event network-changed:b8947813-7d62-4e04-8aa8-8ba174d01def for instance a7260920-be84-4a74-a18e-5b47d53e3db1 on hv-cinder-maste Oct 24 09:05:40.477327 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-33ffceec-2b2f-4207-a9cb-150467a837af req-0d7f7066-203b-4916-92d2-175f34743800 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048026 Oct 24 09:05:40.478514 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 591/1170] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:05:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 09:05:41.791775 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-393aaa02-83bb-481f-a07b-cc068d6abb46 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:41.792301 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-393aaa02-83bb-481f-a07b-cc068d6abb46 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:41.799539 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-393aaa02-83bb-481f-a07b-cc068d6abb46 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:41.800026 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-393aaa02-83bb-481f-a07b-cc068d6abb46 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:41.800523 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-393aaa02-83bb-481f-a07b-cc068d6abb46 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:41.801230 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-393aaa02-83bb-481f-a07b-cc068d6abb46 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:41.801685 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-393aaa02-83bb-481f-a07b-cc068d6abb46 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:41.802114 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-393aaa02-83bb-481f-a07b-cc068d6abb46 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:41.885291 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.objects.instance [None req-393aaa02-83bb-481f-a07b-cc068d6abb46 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lazy-loading 'services' on Instance uuid a7260920-be84-4a74-a18e-5b47d53e3db1 {{(pid=130396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 09:05:41.961342 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-393aaa02-83bb-481f-a07b-cc068d6abb46 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "DELETE /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/os-volume_attachments/8c130732-ebfb-404c-b2b9-691de5349448" status: 202 len: 0 microversion: 2.1 time: 0.173810 Oct 24 09:05:41.962023 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 580/1171] 10.222.0.39 () {58 vars in 1416 bytes} [Mon Oct 24 09:05:41 2022] DELETE /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/os-volume_attachments/8c130732-ebfb-404c-b2b9-691de5349448 => generated 0 bytes in 175 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 09:05:41.972522 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8c70622d-b5a6-490f-9ccf-11dcc06d4757 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:41.975234 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8c70622d-b5a6-490f-9ccf-11dcc06d4757 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:41.984971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c70622d-b5a6-490f-9ccf-11dcc06d4757 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:41.986083 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c70622d-b5a6-490f-9ccf-11dcc06d4757 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:41.986835 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c70622d-b5a6-490f-9ccf-11dcc06d4757 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:41.987770 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c70622d-b5a6-490f-9ccf-11dcc06d4757 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:41.988356 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c70622d-b5a6-490f-9ccf-11dcc06d4757 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:41.988894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8c70622d-b5a6-490f-9ccf-11dcc06d4757 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:42.022023 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8c70622d-b5a6-490f-9ccf-11dcc06d4757 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055183 Oct 24 09:05:42.026487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 592/1172] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 09:05:41 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 09:05:42.493601 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-2d5e47f8-c858-41c6-806e-d33b5bb5a2c6 req-7c5fcdc8-babe-4e70-84a2-d57ef3a577d0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a7260920-be84-4a74-a18e-5b47d53e3db1", "tag": "b8947813-7d62-4e04-8aa8-8ba174d01def"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:05:42.503555 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-2d5e47f8-c858-41c6-806e-d33b5bb5a2c6 req-7c5fcdc8-babe-4e70-84a2-d57ef3a577d0 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:42.504073 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-2d5e47f8-c858-41c6-806e-d33b5bb5a2c6 req-7c5fcdc8-babe-4e70-84a2-d57ef3a577d0 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:42.504572 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-2d5e47f8-c858-41c6-806e-d33b5bb5a2c6 req-7c5fcdc8-babe-4e70-84a2-d57ef3a577d0 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:42.515210 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-2d5e47f8-c858-41c6-806e-d33b5bb5a2c6 req-7c5fcdc8-babe-4e70-84a2-d57ef3a577d0 service nova] Creating event network-changed:b8947813-7d62-4e04-8aa8-8ba174d01def for instance a7260920-be84-4a74-a18e-5b47d53e3db1 on hv-cinder-maste Oct 24 09:05:42.520228 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-2d5e47f8-c858-41c6-806e-d33b5bb5a2c6 req-7c5fcdc8-babe-4e70-84a2-d57ef3a577d0 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030677 Oct 24 09:05:42.520841 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 581/1173] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:05:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:05:43.035379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-585d1c7f-c7a5-437c-b6fb-53fd36ea8d8b tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:43.037599 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-585d1c7f-c7a5-437c-b6fb-53fd36ea8d8b tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:43.044967 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-585d1c7f-c7a5-437c-b6fb-53fd36ea8d8b tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:43.045482 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-585d1c7f-c7a5-437c-b6fb-53fd36ea8d8b tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:43.045920 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-585d1c7f-c7a5-437c-b6fb-53fd36ea8d8b tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:43.046615 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-585d1c7f-c7a5-437c-b6fb-53fd36ea8d8b tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:43.047066 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-585d1c7f-c7a5-437c-b6fb-53fd36ea8d8b tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:43.047485 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-585d1c7f-c7a5-437c-b6fb-53fd36ea8d8b tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:43.078076 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-585d1c7f-c7a5-437c-b6fb-53fd36ea8d8b tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046581 Oct 24 09:05:43.078646 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 593/1174] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 09:05:43 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:05:44.092332 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b6149d6b-1031-49e4-8b99-481cc8151639 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:44.094544 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b6149d6b-1031-49e4-8b99-481cc8151639 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:44.100671 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b6149d6b-1031-49e4-8b99-481cc8151639 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:44.101149 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b6149d6b-1031-49e4-8b99-481cc8151639 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:44.101582 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b6149d6b-1031-49e4-8b99-481cc8151639 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:44.102263 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b6149d6b-1031-49e4-8b99-481cc8151639 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:44.102711 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b6149d6b-1031-49e4-8b99-481cc8151639 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:44.103131 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b6149d6b-1031-49e4-8b99-481cc8151639 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:44.132394 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b6149d6b-1031-49e4-8b99-481cc8151639 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045264 Oct 24 09:05:44.132997 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 582/1175] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 09:05:44 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:05:45.145464 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-3d2ff54a-585f-485f-b1e0-5668eecff1d8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:45.147601 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-3d2ff54a-585f-485f-b1e0-5668eecff1d8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:45.155021 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d2ff54a-585f-485f-b1e0-5668eecff1d8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:45.155497 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d2ff54a-585f-485f-b1e0-5668eecff1d8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:45.155929 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d2ff54a-585f-485f-b1e0-5668eecff1d8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:45.156652 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d2ff54a-585f-485f-b1e0-5668eecff1d8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:45.157100 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d2ff54a-585f-485f-b1e0-5668eecff1d8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:45.157521 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3d2ff54a-585f-485f-b1e0-5668eecff1d8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:45.186144 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-3d2ff54a-585f-485f-b1e0-5668eecff1d8 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044657 Oct 24 09:05:45.186698 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 594/1176] 10.222.0.39 () {58 vars in 1302 bytes} [Mon Oct 24 09:05:45 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 24 09:05:45.197229 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b1d8d026-39b6-41d1-95cd-a7055202f771 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:45.197725 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-b1d8d026-39b6-41d1-95cd-a7055202f771 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:45.204801 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b1d8d026-39b6-41d1-95cd-a7055202f771 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:45.205281 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b1d8d026-39b6-41d1-95cd-a7055202f771 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:45.205715 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b1d8d026-39b6-41d1-95cd-a7055202f771 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:45.206404 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b1d8d026-39b6-41d1-95cd-a7055202f771 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:45.206855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b1d8d026-39b6-41d1-95cd-a7055202f771 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:45.207277 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-b1d8d026-39b6-41d1-95cd-a7055202f771 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:45.281069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-b1d8d026-39b6-41d1-95cd-a7055202f771 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] HTTP exception thrown: Instance a7260920-be84-4a74-a18e-5b47d53e3db1 is not attached to volume 8c130732-ebfb-404c-b2b9-691de5349448 Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b1d8d026-39b6-41d1-95cd-a7055202f771 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Request method failure captured: Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: DELETE /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/os-volume_attachments/8c130732-ebfb-404c-b2b9-691de5349448 HTTP/1.1 Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlRrHMkGNtu1OZ-iTCbLNoRu-zSyleXyI_nwojb6hTfnuEcONRaJrOp4UxBPffU9NYh6XTgD2s86tqbgwVGu4ujhefOFFNwYnxR6r2CoFZ5JmQKyUjqXPPoHdwN3sEeqSnc_mWFDhOSjXZrjhLs48tPtGsALVb8uawJzehlZlQj6Y6Q Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 7e9c4dec37674f73912f0d9272ac2446 Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-TestMinimumBasicScenario-609381336 Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 09:05:45.287668 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 09:05:45.289649 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/7e9c4dec37674f73912f0d9272ac2446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_7e9c4dec37674f73912f0d9272ac2446", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/7e9c4dec37674f73912f0d9272ac2446"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/7e9c4dec37674f73912f0d9272ac2446"}]}] Oct 24 09:05:45.289649 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-TestMinimumBasicScenario-609381336 Oct 24 09:05:45.289649 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 7e9c4dec37674f73912f0d9272ac2446 Oct 24 09:05:45.289649 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-TestMinimumBasicScenario-609381336 Oct 24 09:05:45.289649 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-TestMinimumBasicScenario-609381336-project Oct 24 09:05:45.289649 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 09:05:45.289649 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 09:05:45.289649 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 4f104bf97764426cb67b14bbf39e16be Oct 24 09:05:45.289649 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-TestMinimumBasicScenario-609381336-project Oct 24 09:05:45.289649 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 09:05:45.289649 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance a7260920-be84-4a74-a18e-5b47d53e3db1 is not attached to volume 8c130732-ebfb-404c-b2b9-691de5349448 Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'server_id': 'a7260920-be84-4a74-a18e-5b47d53e3db1', 'id': '8c130732-ebfb-404c-b2b9-691de5349448'} Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 8c130732-ebfb-404c-b2b9-691de5349448. Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:05:45.290650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:05:45.292370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:05:45.292370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 24 09:05:45.292370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 24 09:05:45.292370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a7260920-be84-4a74-a18e-5b47d53e3db1 is not attached to volume 8c130732-ebfb-404c-b2b9-691de5349448 Oct 24 09:05:45.292370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:05:45.292370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:05:45.292370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:05:45.292370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:05:45.292370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:05:45.292370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:05:45.292370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:05:45.292370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:05:45.292370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a7260920-be84-4a74-a18e-5b47d53e3db1 is not attached to volume 8c130732-ebfb-404c-b2b9-691de5349448 Oct 24 09:05:45.292370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:05:45.292370 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-b1d8d026-39b6-41d1-95cd-a7055202f771 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Returning 404 to user: Instance a7260920-be84-4a74-a18e-5b47d53e3db1 is not attached to volume 8c130732-ebfb-404c-b2b9-691de5349448 {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:05:45.293759 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-b1d8d026-39b6-41d1-95cd-a7055202f771 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "DELETE /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/os-volume_attachments/8c130732-ebfb-404c-b2b9-691de5349448" status: 404 len: 154 microversion: 2.1 time: 0.095872 Oct 24 09:05:45.293759 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 583/1177] 10.222.0.39 () {58 vars in 1416 bytes} [Mon Oct 24 09:05:45 2022] DELETE /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1/os-volume_attachments/8c130732-ebfb-404c-b2b9-691de5349448 => generated 154 bytes in 97 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:05:45.470038 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1483599080", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-593019647", "security_groups": [{"name": "tempest-secgroup-smoke-1512102953"}], "block_device_mapping_v2": [{"uuid": "8f8535e7-5370-4d62-bd87-6e2f5749fde0", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "15551789-d406-4f91-85fe-9eee40e4ab51"}]}} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:05:45.586244 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.network.neutron [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] validate_networks() for [('15551789-d406-4f91-85fe-9eee40e4ab51', None, None, None, None, None)] {{(pid=130395) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 24 09:05:45.962290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Flavor limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 24 09:05:45.962709 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Image limits 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 24 09:05:45.963175 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Flavor pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 24 09:05:45.963577 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Image pref 0:0:0 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 24 09:05:45.963984 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.virt.hardware [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130395) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 24 09:05:45.973708 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:45.974187 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:45.974616 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:45.975341 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:45.975786 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:45.976229 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:45.993226 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Getting quotas for project 30742e7b914243df8dd120079560a3de. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 24 09:05:45.998153 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.quota [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Getting quotas for user 3cddd27c37514e74a38430c46cef16fa and project 30742e7b914243df8dd120079560a3de. Resources: {'ram', 'instances', 'cores'} {{(pid=130395) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 24 09:05:46.054871 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Going to run 1 instances... {{(pid=130395) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 24 09:05:46.057058 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='8f8535e7-5370-4d62-bd87-6e2f5749fde0',volume_size=None,volume_type=None)] {{(pid=130395) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 24 09:05:46.186503 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d58ad755-f94f-4bfe-a1f4-00acf556414b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.720297 Oct 24 09:05:46.187998 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 595/1178] 10.222.0.39 () {60 vars in 1147 bytes} [Mon Oct 24 09:05:45 2022] POST /compute/v2.1/servers => generated 406 bytes in 722 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 24 09:05:46.198210 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c359502c-0d0d-49a2-8daf-a15613257cb6 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:46.198819 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c359502c-0d0d-49a2-8daf-a15613257cb6 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:46.254921 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c359502c-0d0d-49a2-8daf-a15613257cb6 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:05:46.263904 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c359502c-0d0d-49a2-8daf-a15613257cb6 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1094 microversion: 2.1 time: 0.069501 Oct 24 09:05:46.264497 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 584/1179] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:05:46 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1094 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:05:46.444658 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-aff83acc-f41b-4a52-88fd-2a70f55466bd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:46.445235 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-aff83acc-f41b-4a52-88fd-2a70f55466bd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:46.452180 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aff83acc-f41b-4a52-88fd-2a70f55466bd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:46.452770 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aff83acc-f41b-4a52-88fd-2a70f55466bd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:46.453290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aff83acc-f41b-4a52-88fd-2a70f55466bd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:46.454102 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aff83acc-f41b-4a52-88fd-2a70f55466bd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:46.454584 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aff83acc-f41b-4a52-88fd-2a70f55466bd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:46.455394 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-aff83acc-f41b-4a52-88fd-2a70f55466bd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:46.482327 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-aff83acc-f41b-4a52-88fd-2a70f55466bd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Going to try to terminate instance {{(pid=130395) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 09:05:46.563465 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-aff83acc-f41b-4a52-88fd-2a70f55466bd tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "DELETE /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 204 len: 0 microversion: 2.1 time: 0.122441 Oct 24 09:05:46.564614 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 596/1180] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 09:05:46 2022] DELETE /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 09:05:46.572947 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f5ca1f92-26d2-4920-bc5d-d0444b7ce4d6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:46.573553 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f5ca1f92-26d2-4920-bc5d-d0444b7ce4d6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:46.580010 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f5ca1f92-26d2-4920-bc5d-d0444b7ce4d6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:46.580529 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f5ca1f92-26d2-4920-bc5d-d0444b7ce4d6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:46.580964 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f5ca1f92-26d2-4920-bc5d-d0444b7ce4d6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:46.581652 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f5ca1f92-26d2-4920-bc5d-d0444b7ce4d6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:46.582102 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f5ca1f92-26d2-4920-bc5d-d0444b7ce4d6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:46.582525 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f5ca1f92-26d2-4920-bc5d-d0444b7ce4d6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:46.707593 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-f5ca1f92-26d2-4920-bc5d-d0444b7ce4d6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:05:46.715215 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f5ca1f92-26d2-4920-bc5d-d0444b7ce4d6 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1553 microversion: 2.1 time: 0.146201 Oct 24 09:05:46.716926 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 585/1181] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:05:46 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1553 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:05:47.287184 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-e659c63d-903b-48bc-bb62-94432fc4e5af tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:47.287808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-e659c63d-903b-48bc-bb62-94432fc4e5af tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:47.294290 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e659c63d-903b-48bc-bb62-94432fc4e5af tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:47.294775 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e659c63d-903b-48bc-bb62-94432fc4e5af tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:47.295220 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e659c63d-903b-48bc-bb62-94432fc4e5af tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:47.295917 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e659c63d-903b-48bc-bb62-94432fc4e5af tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:47.296408 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e659c63d-903b-48bc-bb62-94432fc4e5af tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:47.296839 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-e659c63d-903b-48bc-bb62-94432fc4e5af tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:47.361544 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-e659c63d-903b-48bc-bb62-94432fc4e5af tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:05:47.369115 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-e659c63d-903b-48bc-bb62-94432fc4e5af tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1144 microversion: 2.1 time: 0.086328 Oct 24 09:05:47.369709 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 597/1182] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:05:47 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1144 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:05:47.740151 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0d1d8c8d-97d3-4dac-9e23-89d4f5258aa9 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:47.740777 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-0d1d8c8d-97d3-4dac-9e23-89d4f5258aa9 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:47.747972 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d1d8c8d-97d3-4dac-9e23-89d4f5258aa9 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:47.748738 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d1d8c8d-97d3-4dac-9e23-89d4f5258aa9 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:47.749178 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d1d8c8d-97d3-4dac-9e23-89d4f5258aa9 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:47.749871 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d1d8c8d-97d3-4dac-9e23-89d4f5258aa9 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:47.750326 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d1d8c8d-97d3-4dac-9e23-89d4f5258aa9 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:47.750752 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0d1d8c8d-97d3-4dac-9e23-89d4f5258aa9 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:47.880474 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0d1d8c8d-97d3-4dac-9e23-89d4f5258aa9 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:05:47.888700 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-0d1d8c8d-97d3-4dac-9e23-89d4f5258aa9 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1553 microversion: 2.1 time: 0.152497 Oct 24 09:05:47.889260 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 586/1183] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:05:47 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1553 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:05:48.393250 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-d1723c59-0357-4f8c-b3a1-ef51aab11bfe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:48.394118 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-d1723c59-0357-4f8c-b3a1-ef51aab11bfe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:48.402133 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d1723c59-0357-4f8c-b3a1-ef51aab11bfe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:48.402627 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d1723c59-0357-4f8c-b3a1-ef51aab11bfe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:48.403073 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d1723c59-0357-4f8c-b3a1-ef51aab11bfe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:48.403992 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d1723c59-0357-4f8c-b3a1-ef51aab11bfe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:48.404480 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d1723c59-0357-4f8c-b3a1-ef51aab11bfe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:48.404956 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-d1723c59-0357-4f8c-b3a1-ef51aab11bfe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:48.467993 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-d1723c59-0357-4f8c-b3a1-ef51aab11bfe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:05:48.476745 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-d1723c59-0357-4f8c-b3a1-ef51aab11bfe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1192 microversion: 2.1 time: 0.087275 Oct 24 09:05:48.477340 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 598/1184] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:05:48 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1192 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:05:48.913726 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-58984aa3-2350-46ed-a8a6-7aa0c8cdbf7c tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:48.914709 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-58984aa3-2350-46ed-a8a6-7aa0c8cdbf7c tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:48.922540 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-58984aa3-2350-46ed-a8a6-7aa0c8cdbf7c tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:48.923021 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-58984aa3-2350-46ed-a8a6-7aa0c8cdbf7c tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:48.923455 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-58984aa3-2350-46ed-a8a6-7aa0c8cdbf7c tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:48.924174 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-58984aa3-2350-46ed-a8a6-7aa0c8cdbf7c tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:48.924626 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-58984aa3-2350-46ed-a8a6-7aa0c8cdbf7c tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:48.925046 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-58984aa3-2350-46ed-a8a6-7aa0c8cdbf7c tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:49.085957 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-58984aa3-2350-46ed-a8a6-7aa0c8cdbf7c tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:05:49.095082 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-58984aa3-2350-46ed-a8a6-7aa0c8cdbf7c tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1553 microversion: 2.1 time: 0.186419 Oct 24 09:05:49.103025 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 587/1185] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:05:48 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1553 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:05:49.503960 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-56ef697f-a55d-4e71-b372-78a29c24fe0c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:49.504768 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-56ef697f-a55d-4e71-b372-78a29c24fe0c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:49.513784 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-56ef697f-a55d-4e71-b372-78a29c24fe0c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:49.514545 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-56ef697f-a55d-4e71-b372-78a29c24fe0c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:49.515163 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-56ef697f-a55d-4e71-b372-78a29c24fe0c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:49.516088 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-56ef697f-a55d-4e71-b372-78a29c24fe0c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:49.516742 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-56ef697f-a55d-4e71-b372-78a29c24fe0c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:49.517845 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-56ef697f-a55d-4e71-b372-78a29c24fe0c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:49.561521 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-857360d0-d730-4046-bc94-b03733b53464 req-45007eaa-4bd5-42dc-9f9e-22b047105897 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a7260920-be84-4a74-a18e-5b47d53e3db1", "name": "network-vif-deleted", "tag": "b8947813-7d62-4e04-8aa8-8ba174d01def"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:05:49.576546 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-857360d0-d730-4046-bc94-b03733b53464 req-45007eaa-4bd5-42dc-9f9e-22b047105897 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:49.577477 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-857360d0-d730-4046-bc94-b03733b53464 req-45007eaa-4bd5-42dc-9f9e-22b047105897 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:49.578097 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-857360d0-d730-4046-bc94-b03733b53464 req-45007eaa-4bd5-42dc-9f9e-22b047105897 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:49.591051 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-56ef697f-a55d-4e71-b372-78a29c24fe0c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:05:49.592164 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-857360d0-d730-4046-bc94-b03733b53464 req-45007eaa-4bd5-42dc-9f9e-22b047105897 service nova] Creating event network-vif-deleted:b8947813-7d62-4e04-8aa8-8ba174d01def for instance a7260920-be84-4a74-a18e-5b47d53e3db1 on hv-cinder-maste Oct 24 09:05:49.598731 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-857360d0-d730-4046-bc94-b03733b53464 req-45007eaa-4bd5-42dc-9f9e-22b047105897 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041586 Oct 24 09:05:49.599297 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 588/1186] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:05:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 09:05:49.602007 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-56ef697f-a55d-4e71-b372-78a29c24fe0c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1210 microversion: 2.1 time: 0.104357 Oct 24 09:05:49.602501 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 599/1187] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:05:49 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1210 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:05:50.119770 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c7d688d4-af5f-44d7-9cf3-b96facc09aa7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:50.120561 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c7d688d4-af5f-44d7-9cf3-b96facc09aa7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:50.127695 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c7d688d4-af5f-44d7-9cf3-b96facc09aa7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:50.128641 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c7d688d4-af5f-44d7-9cf3-b96facc09aa7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:50.128855 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c7d688d4-af5f-44d7-9cf3-b96facc09aa7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:50.129843 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c7d688d4-af5f-44d7-9cf3-b96facc09aa7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:50.130471 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c7d688d4-af5f-44d7-9cf3-b96facc09aa7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:50.131219 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c7d688d4-af5f-44d7-9cf3-b96facc09aa7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:50.171838 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c7d688d4-af5f-44d7-9cf3-b96facc09aa7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:50.172859 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c7d688d4-af5f-44d7-9cf3-b96facc09aa7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:50.173254 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c7d688d4-af5f-44d7-9cf3-b96facc09aa7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:50.173713 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.objects.instance [None req-c7d688d4-af5f-44d7-9cf3-b96facc09aa7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lazy-loading 'fault' on Instance uuid a7260920-be84-4a74-a18e-5b47d53e3db1 {{(pid=130396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 24 09:05:50.220205 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c7d688d4-af5f-44d7-9cf3-b96facc09aa7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4f104bf97764426cb67b14bbf39e16be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e9c4dec37674f73912f0d9272ac2446', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:05:50.229334 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c7d688d4-af5f-44d7-9cf3-b96facc09aa7 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 200 len: 1348 microversion: 2.1 time: 0.114022 Oct 24 09:05:50.236375 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 589/1188] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:05:50 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 1348 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:05:50.627212 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-74fa05b4-c040-4785-bc43-46ce8ca274fd tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:50.627831 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-74fa05b4-c040-4785-bc43-46ce8ca274fd tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:50.634218 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-74fa05b4-c040-4785-bc43-46ce8ca274fd tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:50.634703 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-74fa05b4-c040-4785-bc43-46ce8ca274fd tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:50.635198 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-74fa05b4-c040-4785-bc43-46ce8ca274fd tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:50.636002 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-74fa05b4-c040-4785-bc43-46ce8ca274fd tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:50.636507 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-74fa05b4-c040-4785-bc43-46ce8ca274fd tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:50.636923 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-74fa05b4-c040-4785-bc43-46ce8ca274fd tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:50.767918 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-74fa05b4-c040-4785-bc43-46ce8ca274fd tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:05:50.776045 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-74fa05b4-c040-4785-bc43-46ce8ca274fd tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1266 microversion: 2.1 time: 0.152939 Oct 24 09:05:50.776661 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 600/1189] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:05:50 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:05:50.803529 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-d9553f8e-50f9-4566-8577-a7bc9ecccf70 req-94e8adb3-0b07-4c1a-a373-60cbb2d7ab40 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0aa6b4f8-4ef2-4600-8684-6a7ea6812917", "tag": "66fd3d85-3e82-478a-b0aa-2baf2f290298"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:05:50.815532 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-d9553f8e-50f9-4566-8577-a7bc9ecccf70 req-94e8adb3-0b07-4c1a-a373-60cbb2d7ab40 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:50.816430 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-d9553f8e-50f9-4566-8577-a7bc9ecccf70 req-94e8adb3-0b07-4c1a-a373-60cbb2d7ab40 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:50.817047 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-d9553f8e-50f9-4566-8577-a7bc9ecccf70 req-94e8adb3-0b07-4c1a-a373-60cbb2d7ab40 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:50.828885 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-d9553f8e-50f9-4566-8577-a7bc9ecccf70 req-94e8adb3-0b07-4c1a-a373-60cbb2d7ab40 service nova] Creating event network-changed:66fd3d85-3e82-478a-b0aa-2baf2f290298 for instance 0aa6b4f8-4ef2-4600-8684-6a7ea6812917 on hv-cinder-maste Oct 24 09:05:50.833883 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-d9553f8e-50f9-4566-8577-a7bc9ecccf70 req-94e8adb3-0b07-4c1a-a373-60cbb2d7ab40 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034329 Oct 24 09:05:50.834469 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 590/1190] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:05:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:05:51.253963 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-cbf5a616-94a4-4882-8d23-ff435a206ed4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:51.254584 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-cbf5a616-94a4-4882-8d23-ff435a206ed4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] [instance: a7260920-be84-4a74-a18e-5b47d53e3db1] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:51.260823 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cbf5a616-94a4-4882-8d23-ff435a206ed4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:51.261382 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cbf5a616-94a4-4882-8d23-ff435a206ed4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:51.261837 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cbf5a616-94a4-4882-8d23-ff435a206ed4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:51.263199 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cbf5a616-94a4-4882-8d23-ff435a206ed4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:51.263658 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cbf5a616-94a4-4882-8d23-ff435a206ed4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:51.264089 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-cbf5a616-94a4-4882-8d23-ff435a206ed4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:51.280304 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-cbf5a616-94a4-4882-8d23-ff435a206ed4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] HTTP exception thrown: Instance a7260920-be84-4a74-a18e-5b47d53e3db1 could not be found. Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-cbf5a616-94a4-4882-8d23-ff435a206ed4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Request method failure captured: Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 HTTP/1.1 Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlRrHMkGNtu1OZ-iTCbLNoRu-zSyleXyI_nwojb6hTfnuEcONRaJrOp4UxBPffU9NYh6XTgD2s86tqbgwVGu4ujhefOFFNwYnxR6r2CoFZ5JmQKyUjqXPPoHdwN3sEeqSnc_mWFDhOSjXZrjhLs48tPtGsALVb8uawJzehlZlQj6Y6Q Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: 7e9c4dec37674f73912f0d9272ac2446 Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-TestMinimumBasicScenario-609381336 Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:05:51.282041 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:05:51.283808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/7e9c4dec37674f73912f0d9272ac2446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_7e9c4dec37674f73912f0d9272ac2446", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/7e9c4dec37674f73912f0d9272ac2446"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/7e9c4dec37674f73912f0d9272ac2446"}]}] Oct 24 09:05:51.283808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-TestMinimumBasicScenario-609381336 Oct 24 09:05:51.283808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: 7e9c4dec37674f73912f0d9272ac2446 Oct 24 09:05:51.283808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-TestMinimumBasicScenario-609381336 Oct 24 09:05:51.283808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-TestMinimumBasicScenario-609381336-project Oct 24 09:05:51.283808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:05:51.283808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:05:51.283808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: 4f104bf97764426cb67b14bbf39e16be Oct 24 09:05:51.283808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-TestMinimumBasicScenario-609381336-project Oct 24 09:05:51.283808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 09:05:51.283808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance a7260920-be84-4a74-a18e-5b47d53e3db1 could not be found. Oct 24 09:05:51.283808 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': 'a7260920-be84-4a74-a18e-5b47d53e3db1'} Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a7260920-be84-4a74-a18e-5b47d53e3db1 could not be found. Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:05:51.287667 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a7260920-be84-4a74-a18e-5b47d53e3db1 could not be found. Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a7260920-be84-4a74-a18e-5b47d53e3db1 could not be found. Oct 24 09:05:51.289393 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:05:51.290849 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-cbf5a616-94a4-4882-8d23-ff435a206ed4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Returning 404 to user: Instance a7260920-be84-4a74-a18e-5b47d53e3db1 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:05:51.290849 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-cbf5a616-94a4-4882-8d23-ff435a206ed4 tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1" status: 404 len: 111 microversion: 2.1 time: 0.033438 Oct 24 09:05:51.290849 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 601/1191] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:05:51 2022] GET /compute/v2.1/servers/a7260920-be84-4a74-a18e-5b47d53e3db1 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 24 09:05:51.292723 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-151242f2-4ff5-4374-9d9d-e613188930fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] Calling method '.version_select at 0x7fdfedcc0e50>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:51.315407 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-151242f2-4ff5-4374-9d9d-e613188930fc tempest-TestMinimumBasicScenario-609381336 tempest-TestMinimumBasicScenario-609381336-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-624156899" status: 202 len: 0 microversion: 2.1 time: 0.026820 Oct 24 09:05:51.316577 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 591/1192] 10.222.0.39 () {58 vars in 1269 bytes} [Mon Oct 24 09:05:51 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-624156899 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 24 09:05:51.801000 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-dd3370f4-bf1f-4493-a315-e5dbc2f5fd93 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:51.801675 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-dd3370f4-bf1f-4493-a315-e5dbc2f5fd93 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:51.808815 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dd3370f4-bf1f-4493-a315-e5dbc2f5fd93 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:51.809790 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dd3370f4-bf1f-4493-a315-e5dbc2f5fd93 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:51.810416 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dd3370f4-bf1f-4493-a315-e5dbc2f5fd93 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:51.811135 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dd3370f4-bf1f-4493-a315-e5dbc2f5fd93 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:51.811596 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dd3370f4-bf1f-4493-a315-e5dbc2f5fd93 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:51.812025 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-dd3370f4-bf1f-4493-a315-e5dbc2f5fd93 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:52.197891 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-dd3370f4-bf1f-4493-a315-e5dbc2f5fd93 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:05:52.206365 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-dd3370f4-bf1f-4493-a315-e5dbc2f5fd93 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1266 microversion: 2.1 time: 0.409354 Oct 24 09:05:52.206938 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 602/1193] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:05:51 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1266 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:05:53.229980 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-e0e56c9e-7018-4699-8a9a-5113db9fdd07 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:53.231280 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-e0e56c9e-7018-4699-8a9a-5113db9fdd07 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:53.239189 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0e56c9e-7018-4699-8a9a-5113db9fdd07 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:53.239683 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0e56c9e-7018-4699-8a9a-5113db9fdd07 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:53.240115 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0e56c9e-7018-4699-8a9a-5113db9fdd07 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:53.240897 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0e56c9e-7018-4699-8a9a-5113db9fdd07 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:53.241539 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0e56c9e-7018-4699-8a9a-5113db9fdd07 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:53.242132 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-e0e56c9e-7018-4699-8a9a-5113db9fdd07 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:53.470463 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-e0e56c9e-7018-4699-8a9a-5113db9fdd07 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:05:53.478577 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-e0e56c9e-7018-4699-8a9a-5113db9fdd07 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1266 microversion: 2.1 time: 0.252964 Oct 24 09:05:53.479462 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 592/1194] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:05:53 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1266 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 09:05:54.501241 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-843c9f8f-e62c-4034-858e-596c4a1d7f2d tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:54.501989 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-843c9f8f-e62c-4034-858e-596c4a1d7f2d tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:54.508473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-843c9f8f-e62c-4034-858e-596c4a1d7f2d tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:54.508959 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-843c9f8f-e62c-4034-858e-596c4a1d7f2d tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:54.509398 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-843c9f8f-e62c-4034-858e-596c4a1d7f2d tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:54.510099 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-843c9f8f-e62c-4034-858e-596c4a1d7f2d tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:54.510560 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-843c9f8f-e62c-4034-858e-596c4a1d7f2d tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:54.510987 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-843c9f8f-e62c-4034-858e-596c4a1d7f2d tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:54.756430 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-843c9f8f-e62c-4034-858e-596c4a1d7f2d tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:05:54.764753 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-843c9f8f-e62c-4034-858e-596c4a1d7f2d tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1266 microversion: 2.1 time: 0.267691 Oct 24 09:05:54.765325 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 603/1195] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:05:54 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1266 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:05:55.790412 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-3f2861e7-69d8-41d4-94e3-dcd5bc4d2f83 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:55.791850 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-3f2861e7-69d8-41d4-94e3-dcd5bc4d2f83 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:55.798403 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3f2861e7-69d8-41d4-94e3-dcd5bc4d2f83 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:55.798889 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3f2861e7-69d8-41d4-94e3-dcd5bc4d2f83 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:55.799329 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3f2861e7-69d8-41d4-94e3-dcd5bc4d2f83 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:55.800039 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3f2861e7-69d8-41d4-94e3-dcd5bc4d2f83 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:55.800519 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3f2861e7-69d8-41d4-94e3-dcd5bc4d2f83 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:55.800946 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-3f2861e7-69d8-41d4-94e3-dcd5bc4d2f83 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:56.014974 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-3f2861e7-69d8-41d4-94e3-dcd5bc4d2f83 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:05:56.022862 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-3f2861e7-69d8-41d4-94e3-dcd5bc4d2f83 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1266 microversion: 2.1 time: 0.236242 Oct 24 09:05:56.023502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 593/1196] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:05:55 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1266 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:05:57.055047 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-f0109af6-94cf-451d-aff8-a6f6ff2be1a0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:57.055697 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-f0109af6-94cf-451d-aff8-a6f6ff2be1a0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:57.058332 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=2 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:05:57.059039 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell0:SELECT=6 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:05:57.066414 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f0109af6-94cf-451d-aff8-a6f6ff2be1a0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:57.066940 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f0109af6-94cf-451d-aff8-a6f6ff2be1a0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:57.067403 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f0109af6-94cf-451d-aff8-a6f6ff2be1a0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:57.068106 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f0109af6-94cf-451d-aff8-a6f6ff2be1a0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:57.068579 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f0109af6-94cf-451d-aff8-a6f6ff2be1a0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:57.069007 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-f0109af6-94cf-451d-aff8-a6f6ff2be1a0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:57.244367 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-f0109af6-94cf-451d-aff8-a6f6ff2be1a0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:05:57.253129 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-f0109af6-94cf-451d-aff8-a6f6ff2be1a0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1266 microversion: 2.1 time: 0.202826 Oct 24 09:05:57.254036 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 604/1197] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:05:57 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1266 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:05:58.282022 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-fceb3a54-5064-4736-9e57-743da6502db7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:58.283219 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-fceb3a54-5064-4736-9e57-743da6502db7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:58.289366 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell0:SELECT=2 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:05:58.292885 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fceb3a54-5064-4736-9e57-743da6502db7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:58.293415 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fceb3a54-5064-4736-9e57-743da6502db7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:58.295193 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fceb3a54-5064-4736-9e57-743da6502db7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:58.296444 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fceb3a54-5064-4736-9e57-743da6502db7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:58.297321 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fceb3a54-5064-4736-9e57-743da6502db7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:58.298088 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-fceb3a54-5064-4736-9e57-743da6502db7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:58.492365 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-fceb3a54-5064-4736-9e57-743da6502db7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:05:58.501110 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-fceb3a54-5064-4736-9e57-743da6502db7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1266 microversion: 2.1 time: 0.227983 Oct 24 09:05:58.501685 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 594/1198] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:05:58 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1266 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:05:59.524187 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-17ee3c04-1f9d-478d-b2e4-3b579be9eeeb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:05:59.524813 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-17ee3c04-1f9d-478d-b2e4-3b579be9eeeb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:05:59.532484 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-17ee3c04-1f9d-478d-b2e4-3b579be9eeeb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:59.532977 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-17ee3c04-1f9d-478d-b2e4-3b579be9eeeb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:59.533415 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-17ee3c04-1f9d-478d-b2e4-3b579be9eeeb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:59.534160 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-17ee3c04-1f9d-478d-b2e4-3b579be9eeeb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:05:59.534637 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-17ee3c04-1f9d-478d-b2e4-3b579be9eeeb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:05:59.535065 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-17ee3c04-1f9d-478d-b2e4-3b579be9eeeb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:05:59.674458 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-17ee3c04-1f9d-478d-b2e4-3b579be9eeeb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:05:59.682200 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-17ee3c04-1f9d-478d-b2e4-3b579be9eeeb tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1266 microversion: 2.1 time: 0.161978 Oct 24 09:05:59.682761 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 605/1199] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:05:59 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:06:00.705920 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-372f422c-4410-4e28-b82c-2afa5e2920f9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:06:00.706528 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-372f422c-4410-4e28-b82c-2afa5e2920f9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:06:00.713008 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-372f422c-4410-4e28-b82c-2afa5e2920f9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:06:00.713487 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-372f422c-4410-4e28-b82c-2afa5e2920f9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:06:00.713919 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-372f422c-4410-4e28-b82c-2afa5e2920f9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:06:00.714601 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-372f422c-4410-4e28-b82c-2afa5e2920f9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:06:00.715048 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-372f422c-4410-4e28-b82c-2afa5e2920f9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:06:00.715933 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-372f422c-4410-4e28-b82c-2afa5e2920f9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:06:00.848676 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-372f422c-4410-4e28-b82c-2afa5e2920f9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:06:00.857361 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-372f422c-4410-4e28-b82c-2afa5e2920f9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1266 microversion: 2.1 time: 0.154660 Oct 24 09:06:00.857922 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 595/1200] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:06:00 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1266 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:06:01.881027 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-b9c21964-da7f-4cdf-bdfd-555a065a5900 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:06:01.881691 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-b9c21964-da7f-4cdf-bdfd-555a065a5900 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:06:01.888600 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b9c21964-da7f-4cdf-bdfd-555a065a5900 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:06:01.889086 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b9c21964-da7f-4cdf-bdfd-555a065a5900 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:06:01.889526 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b9c21964-da7f-4cdf-bdfd-555a065a5900 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:06:01.890219 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b9c21964-da7f-4cdf-bdfd-555a065a5900 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:06:01.890673 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b9c21964-da7f-4cdf-bdfd-555a065a5900 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:06:01.891099 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-b9c21964-da7f-4cdf-bdfd-555a065a5900 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:06:02.025355 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-b9c21964-da7f-4cdf-bdfd-555a065a5900 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:06:02.034720 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-b9c21964-da7f-4cdf-bdfd-555a065a5900 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1266 microversion: 2.1 time: 0.157687 Oct 24 09:06:02.035766 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 606/1201] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:06:01 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1266 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:06:03.058664 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-8dab9b08-ab67-4c18-9db3-059c798ed4a5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:06:03.059913 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-8dab9b08-ab67-4c18-9db3-059c798ed4a5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:06:03.066535 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8dab9b08-ab67-4c18-9db3-059c798ed4a5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:06:03.067095 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8dab9b08-ab67-4c18-9db3-059c798ed4a5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:06:03.067531 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8dab9b08-ab67-4c18-9db3-059c798ed4a5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:06:03.068255 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8dab9b08-ab67-4c18-9db3-059c798ed4a5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:06:03.068710 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8dab9b08-ab67-4c18-9db3-059c798ed4a5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:06:03.069132 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-8dab9b08-ab67-4c18-9db3-059c798ed4a5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:06:03.209436 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-8dab9b08-ab67-4c18-9db3-059c798ed4a5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:06:03.217358 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-8dab9b08-ab67-4c18-9db3-059c798ed4a5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1266 microversion: 2.1 time: 0.162744 Oct 24 09:06:03.217934 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 596/1202] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:06:03 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1266 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:06:03.595387 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-ea0d288d-cd82-4b6b-b7be-af8ce21c13a1 req-d724a206-0ba4-4595-94b1-6fe36368c516 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0aa6b4f8-4ef2-4600-8684-6a7ea6812917", "name": "network-vif-plugged", "status": "completed", "tag": "66fd3d85-3e82-478a-b0aa-2baf2f290298"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:06:03.608396 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-ea0d288d-cd82-4b6b-b7be-af8ce21c13a1 req-d724a206-0ba4-4595-94b1-6fe36368c516 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:06:03.609136 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-ea0d288d-cd82-4b6b-b7be-af8ce21c13a1 req-d724a206-0ba4-4595-94b1-6fe36368c516 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:06:03.609798 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-ea0d288d-cd82-4b6b-b7be-af8ce21c13a1 req-d724a206-0ba4-4595-94b1-6fe36368c516 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:06:03.622694 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-ea0d288d-cd82-4b6b-b7be-af8ce21c13a1 req-d724a206-0ba4-4595-94b1-6fe36368c516 service nova] Creating event network-vif-plugged:66fd3d85-3e82-478a-b0aa-2baf2f290298 for instance 0aa6b4f8-4ef2-4600-8684-6a7ea6812917 on hv-cinder-maste Oct 24 09:06:03.628690 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-ea0d288d-cd82-4b6b-b7be-af8ce21c13a1 req-d724a206-0ba4-4595-94b1-6fe36368c516 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037012 Oct 24 09:06:03.629531 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 607/1203] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:06:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:06:04.241474 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-0315ec20-55a2-46ef-83f6-3e4842b98d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:06:04.242046 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-0315ec20-55a2-46ef-83f6-3e4842b98d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:06:04.249437 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0315ec20-55a2-46ef-83f6-3e4842b98d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:06:04.249920 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0315ec20-55a2-46ef-83f6-3e4842b98d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:06:04.250355 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0315ec20-55a2-46ef-83f6-3e4842b98d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:06:04.251054 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0315ec20-55a2-46ef-83f6-3e4842b98d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:06:04.251504 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0315ec20-55a2-46ef-83f6-3e4842b98d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:06:04.252044 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-0315ec20-55a2-46ef-83f6-3e4842b98d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:06:04.391383 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-0315ec20-55a2-46ef-83f6-3e4842b98d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:06:04.398954 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-0315ec20-55a2-46ef-83f6-3e4842b98d05 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1266 microversion: 2.1 time: 0.161465 Oct 24 09:06:04.399518 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 597/1204] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:06:04 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:06:05.424311 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-3c3d06f8-8a50-493d-8165-86e12bf084ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:06:05.425050 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-3c3d06f8-8a50-493d-8165-86e12bf084ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:06:05.431139 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3c3d06f8-8a50-493d-8165-86e12bf084ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:06:05.431619 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3c3d06f8-8a50-493d-8165-86e12bf084ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:06:05.432051 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3c3d06f8-8a50-493d-8165-86e12bf084ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:06:05.432758 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3c3d06f8-8a50-493d-8165-86e12bf084ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:06:05.433208 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3c3d06f8-8a50-493d-8165-86e12bf084ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:06:05.433666 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-3c3d06f8-8a50-493d-8165-86e12bf084ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:06:05.454201 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:06:05.577524 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-3c3d06f8-8a50-493d-8165-86e12bf084ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:06:05.585124 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-3c3d06f8-8a50-493d-8165-86e12bf084ca tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1266 microversion: 2.1 time: 0.166705 Oct 24 09:06:05.585683 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 608/1205] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:06:05 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1266 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:06:06.608200 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-d7c2343e-d530-4075-a8f2-c8fb45bfba11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:06:06.608828 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-d7c2343e-d530-4075-a8f2-c8fb45bfba11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:06:06.615513 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7c2343e-d530-4075-a8f2-c8fb45bfba11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:06:06.615992 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7c2343e-d530-4075-a8f2-c8fb45bfba11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:06:06.616455 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7c2343e-d530-4075-a8f2-c8fb45bfba11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:06:06.617145 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7c2343e-d530-4075-a8f2-c8fb45bfba11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:06:06.617649 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7c2343e-d530-4075-a8f2-c8fb45bfba11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:06:06.618121 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-d7c2343e-d530-4075-a8f2-c8fb45bfba11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:06:06.749294 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-d7c2343e-d530-4075-a8f2-c8fb45bfba11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:06:06.752705 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:06:06.758122 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-d7c2343e-d530-4075-a8f2-c8fb45bfba11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1453 microversion: 2.1 time: 0.154633 Oct 24 09:06:06.758696 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 598/1206] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:06:06 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1453 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:06:06.780472 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1bf8dee1-284a-4cfb-9af8-42bafae8476b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:06:06.781088 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1bf8dee1-284a-4cfb-9af8-42bafae8476b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:06:06.787448 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1bf8dee1-284a-4cfb-9af8-42bafae8476b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:06:06.788002 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1bf8dee1-284a-4cfb-9af8-42bafae8476b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:06:06.788544 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1bf8dee1-284a-4cfb-9af8-42bafae8476b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:06:06.789301 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1bf8dee1-284a-4cfb-9af8-42bafae8476b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:06:06.789754 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1bf8dee1-284a-4cfb-9af8-42bafae8476b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:06:06.790230 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1bf8dee1-284a-4cfb-9af8-42bafae8476b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:06:06.921096 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1bf8dee1-284a-4cfb-9af8-42bafae8476b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:06:06.928743 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1bf8dee1-284a-4cfb-9af8-42bafae8476b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1453 microversion: 2.1 time: 0.152297 Oct 24 09:06:06.929310 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 609/1207] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:06:06 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1453 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:06:08.994864 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-4140a622-41c8-4610-bdb9-d84cb4bd65e4 req-92a72f33-7164-4c2a-9d66-6ec744568d98 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0aa6b4f8-4ef2-4600-8684-6a7ea6812917", "tag": "66fd3d85-3e82-478a-b0aa-2baf2f290298"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:06:09.006091 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-4140a622-41c8-4610-bdb9-d84cb4bd65e4 req-92a72f33-7164-4c2a-9d66-6ec744568d98 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:06:09.006562 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-4140a622-41c8-4610-bdb9-d84cb4bd65e4 req-92a72f33-7164-4c2a-9d66-6ec744568d98 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:06:09.006985 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-4140a622-41c8-4610-bdb9-d84cb4bd65e4 req-92a72f33-7164-4c2a-9d66-6ec744568d98 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:06:09.018788 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-4140a622-41c8-4610-bdb9-d84cb4bd65e4 req-92a72f33-7164-4c2a-9d66-6ec744568d98 service nova] Creating event network-changed:66fd3d85-3e82-478a-b0aa-2baf2f290298 for instance 0aa6b4f8-4ef2-4600-8684-6a7ea6812917 on hv-cinder-maste Oct 24 09:06:09.023723 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-4140a622-41c8-4610-bdb9-d84cb4bd65e4 req-92a72f33-7164-4c2a-9d66-6ec744568d98 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033044 Oct 24 09:06:09.024339 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 599/1208] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:06:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:07:58.628452 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_api:SELECT=56,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:07:58.629630 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG dbcounter [-] [130395] Writing DB stats nova_cell1:SELECT=8 {{(pid=130395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:07:58.633574 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_api:SELECT=47,nova_api:DELETE=1 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:07:58.634909 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-7281bca0-6aaf-4d31-bf79-bd7119f2522b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:07:58.635241 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG dbcounter [-] [130396] Writing DB stats nova_cell1:SELECT=2 {{(pid=130396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:07:58.635557 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7281bca0-6aaf-4d31-bf79-bd7119f2522b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:07:58.642272 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-d4f73aab-c5d0-42b1-a142-5648942e17ea req-4455f2e6-2a2f-41a3-8652-ff7027dd300b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0aa6b4f8-4ef2-4600-8684-6a7ea6812917", "tag": "66fd3d85-3e82-478a-b0aa-2baf2f290298"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:07:58.643521 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7281bca0-6aaf-4d31-bf79-bd7119f2522b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:07:58.644008 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7281bca0-6aaf-4d31-bf79-bd7119f2522b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:07:58.644480 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7281bca0-6aaf-4d31-bf79-bd7119f2522b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:07:58.645196 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7281bca0-6aaf-4d31-bf79-bd7119f2522b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:07:58.645682 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7281bca0-6aaf-4d31-bf79-bd7119f2522b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:07:58.646109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-7281bca0-6aaf-4d31-bf79-bd7119f2522b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:07:58.656815 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-d4f73aab-c5d0-42b1-a142-5648942e17ea req-4455f2e6-2a2f-41a3-8652-ff7027dd300b service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:07:58.657766 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-d4f73aab-c5d0-42b1-a142-5648942e17ea req-4455f2e6-2a2f-41a3-8652-ff7027dd300b service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:07:58.658588 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-d4f73aab-c5d0-42b1-a142-5648942e17ea req-4455f2e6-2a2f-41a3-8652-ff7027dd300b service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:07:58.671881 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-d4f73aab-c5d0-42b1-a142-5648942e17ea req-4455f2e6-2a2f-41a3-8652-ff7027dd300b service nova] Creating event network-changed:66fd3d85-3e82-478a-b0aa-2baf2f290298 for instance 0aa6b4f8-4ef2-4600-8684-6a7ea6812917 on hv-cinder-maste Oct 24 09:07:58.673008 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-7281bca0-6aaf-4d31-bf79-bd7119f2522b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Going to try to terminate instance {{(pid=130395) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 09:07:58.677421 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-d4f73aab-c5d0-42b1-a142-5648942e17ea req-4455f2e6-2a2f-41a3-8652-ff7027dd300b service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046400 Oct 24 09:07:58.678402 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 600/1209] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:07:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:07:58.757492 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-7281bca0-6aaf-4d31-bf79-bd7119f2522b tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "DELETE /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 204 len: 0 microversion: 2.1 time: 0.130181 Oct 24 09:07:58.761102 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 610/1210] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 09:07:58 2022] DELETE /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 09:07:58.766746 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-6b89e35b-b96c-47c1-b7de-ea80b55f85d9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:07:58.767460 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-6b89e35b-b96c-47c1-b7de-ea80b55f85d9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:07:58.773804 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b89e35b-b96c-47c1-b7de-ea80b55f85d9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:07:58.774299 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b89e35b-b96c-47c1-b7de-ea80b55f85d9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:07:58.774739 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b89e35b-b96c-47c1-b7de-ea80b55f85d9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:07:58.775809 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b89e35b-b96c-47c1-b7de-ea80b55f85d9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:07:58.775936 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b89e35b-b96c-47c1-b7de-ea80b55f85d9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:07:58.776351 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-6b89e35b-b96c-47c1-b7de-ea80b55f85d9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:07:58.913985 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-6b89e35b-b96c-47c1-b7de-ea80b55f85d9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:07:58.923024 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-6b89e35b-b96c-47c1-b7de-ea80b55f85d9 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1578 microversion: 2.1 time: 0.160575 Oct 24 09:07:58.923583 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 601/1211] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:07:58 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1578 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:07:59.947165 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-833b579d-ca41-4466-a764-32c2e63c3cb0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:07:59.947877 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-833b579d-ca41-4466-a764-32c2e63c3cb0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:07:59.955352 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-833b579d-ca41-4466-a764-32c2e63c3cb0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:07:59.955942 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-833b579d-ca41-4466-a764-32c2e63c3cb0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:07:59.956310 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-833b579d-ca41-4466-a764-32c2e63c3cb0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:07:59.957016 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-833b579d-ca41-4466-a764-32c2e63c3cb0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:07:59.957472 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-833b579d-ca41-4466-a764-32c2e63c3cb0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:07:59.957898 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-833b579d-ca41-4466-a764-32c2e63c3cb0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:00.102474 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-833b579d-ca41-4466-a764-32c2e63c3cb0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:08:00.111033 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-833b579d-ca41-4466-a764-32c2e63c3cb0 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1578 microversion: 2.1 time: 0.168243 Oct 24 09:08:00.111612 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 611/1212] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:07:59 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1578 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:08:01.059234 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-6b6d6e7a-b7e1-4e19-a1bf-5c9feba3d8f3 req-257f6696-725d-4196-ac8f-6dbd8bf9153d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0aa6b4f8-4ef2-4600-8684-6a7ea6812917", "name": "network-vif-deleted", "tag": "66fd3d85-3e82-478a-b0aa-2baf2f290298"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:08:01.069171 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-6b6d6e7a-b7e1-4e19-a1bf-5c9feba3d8f3 req-257f6696-725d-4196-ac8f-6dbd8bf9153d service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:01.069643 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-6b6d6e7a-b7e1-4e19-a1bf-5c9feba3d8f3 req-257f6696-725d-4196-ac8f-6dbd8bf9153d service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:01.070069 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-6b6d6e7a-b7e1-4e19-a1bf-5c9feba3d8f3 req-257f6696-725d-4196-ac8f-6dbd8bf9153d service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:01.082335 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-6b6d6e7a-b7e1-4e19-a1bf-5c9feba3d8f3 req-257f6696-725d-4196-ac8f-6dbd8bf9153d service nova] Creating event network-vif-deleted:66fd3d85-3e82-478a-b0aa-2baf2f290298 for instance 0aa6b4f8-4ef2-4600-8684-6a7ea6812917 on hv-cinder-maste Oct 24 09:08:01.087300 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-6b6d6e7a-b7e1-4e19-a1bf-5c9feba3d8f3 req-257f6696-725d-4196-ac8f-6dbd8bf9153d service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032258 Oct 24 09:08:01.088149 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 602/1213] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:08:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 09:08:01.137143 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-969907c3-2b3a-41a6-b8e3-16949c111f11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:08:01.138005 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-969907c3-2b3a-41a6-b8e3-16949c111f11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:08:01.146005 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-969907c3-2b3a-41a6-b8e3-16949c111f11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:01.146674 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-969907c3-2b3a-41a6-b8e3-16949c111f11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:01.147298 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-969907c3-2b3a-41a6-b8e3-16949c111f11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:01.148104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-969907c3-2b3a-41a6-b8e3-16949c111f11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:01.148665 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-969907c3-2b3a-41a6-b8e3-16949c111f11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:01.149107 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-969907c3-2b3a-41a6-b8e3-16949c111f11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:01.214806 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-969907c3-2b3a-41a6-b8e3-16949c111f11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:08:01.222537 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-969907c3-2b3a-41a6-b8e3-16949c111f11 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 200 len: 1225 microversion: 2.1 time: 0.090218 Oct 24 09:08:01.223104 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 612/1214] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:08:01 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 1225 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:08:02.246927 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a130795c-8f60-4b95-824c-65cce7a4add5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:08:02.247562 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-a130795c-8f60-4b95-824c-65cce7a4add5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 0aa6b4f8-4ef2-4600-8684-6a7ea6812917] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:08:02.254054 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a130795c-8f60-4b95-824c-65cce7a4add5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:02.254534 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a130795c-8f60-4b95-824c-65cce7a4add5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:02.254968 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a130795c-8f60-4b95-824c-65cce7a4add5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:02.255657 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a130795c-8f60-4b95-824c-65cce7a4add5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:02.256108 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a130795c-8f60-4b95-824c-65cce7a4add5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:02.256555 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-a130795c-8f60-4b95-824c-65cce7a4add5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:02.269551 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-a130795c-8f60-4b95-824c-65cce7a4add5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] HTTP exception thrown: Instance 0aa6b4f8-4ef2-4600-8684-6a7ea6812917 could not be found. Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a130795c-8f60-4b95-824c-65cce7a4add5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Request method failure captured: Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 HTTP/1.1 Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlQLoEuIeuhEpW8bsRGnfBcSL-MruEuqIHx01hIOb-lCoWsb18wEa-RL0_ZfZgzNX7tLN6J7zRPamBYUN5Q2h22RCUYnOaAbGpXYsIe862ctqswbh9eXLbV__Gfvp2TY03EtBVsWLXdYyHLJmVeYSUmfNZChusOQVadnrf3HbMtegdk Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 30742e7b914243df8dd120079560a3de Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-TestVolumeBootPattern-2039901937 Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 09:08:02.271144 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 09:08:02.272971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/30742e7b914243df8dd120079560a3de"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_30742e7b914243df8dd120079560a3de", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/30742e7b914243df8dd120079560a3de"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/30742e7b914243df8dd120079560a3de"}]}] Oct 24 09:08:02.272971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-TestVolumeBootPattern-2039901937 Oct 24 09:08:02.272971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 30742e7b914243df8dd120079560a3de Oct 24 09:08:02.272971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-TestVolumeBootPattern-2039901937 Oct 24 09:08:02.272971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-TestVolumeBootPattern-2039901937-project Oct 24 09:08:02.272971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 09:08:02.272971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 09:08:02.272971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 3cddd27c37514e74a38430c46cef16fa Oct 24 09:08:02.272971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-TestVolumeBootPattern-2039901937-project Oct 24 09:08:02.272971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 09:08:02.272971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance 0aa6b4f8-4ef2-4600-8684-6a7ea6812917 could not be found. Oct 24 09:08:02.272971 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': '0aa6b4f8-4ef2-4600-8684-6a7ea6812917'} Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0aa6b4f8-4ef2-4600-8684-6a7ea6812917 could not be found. Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:08:02.274019 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0aa6b4f8-4ef2-4600-8684-6a7ea6812917 could not be found. Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0aa6b4f8-4ef2-4600-8684-6a7ea6812917 could not be found. Oct 24 09:08:02.275708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:08:02.277857 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-a130795c-8f60-4b95-824c-65cce7a4add5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Returning 404 to user: Instance 0aa6b4f8-4ef2-4600-8684-6a7ea6812917 could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:08:02.277857 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-a130795c-8f60-4b95-824c-65cce7a4add5 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917" status: 404 len: 111 microversion: 2.1 time: 0.031085 Oct 24 09:08:02.277857 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 603/1215] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:08:02 2022] GET /compute/v2.1/servers/0aa6b4f8-4ef2-4600-8684-6a7ea6812917 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:08:05.314939 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-8cbfada1-4ab9-4dc6-8da6-89dd93e8892c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:08:05.315506 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8cbfada1-4ab9-4dc6-8da6-89dd93e8892c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:08:05.322157 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8cbfada1-4ab9-4dc6-8da6-89dd93e8892c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:05.322643 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8cbfada1-4ab9-4dc6-8da6-89dd93e8892c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:05.323076 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8cbfada1-4ab9-4dc6-8da6-89dd93e8892c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:05.323988 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8cbfada1-4ab9-4dc6-8da6-89dd93e8892c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:05.324807 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8cbfada1-4ab9-4dc6-8da6-89dd93e8892c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:05.325384 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-8cbfada1-4ab9-4dc6-8da6-89dd93e8892c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:05.351421 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-8cbfada1-4ab9-4dc6-8da6-89dd93e8892c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Going to try to terminate instance {{(pid=130395) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 24 09:08:05.435044 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-8cbfada1-4ab9-4dc6-8da6-89dd93e8892c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "DELETE /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 204 len: 0 microversion: 2.1 time: 0.123721 Oct 24 09:08:05.436160 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 613/1216] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 09:08:05 2022] DELETE /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 24 09:08:05.446253 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-1a578b8f-486a-4dd4-a262-7a2e5f14b1fe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:08:05.447015 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-1a578b8f-486a-4dd4-a262-7a2e5f14b1fe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:08:05.453971 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1a578b8f-486a-4dd4-a262-7a2e5f14b1fe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:05.454465 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1a578b8f-486a-4dd4-a262-7a2e5f14b1fe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:05.454932 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1a578b8f-486a-4dd4-a262-7a2e5f14b1fe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:05.455639 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1a578b8f-486a-4dd4-a262-7a2e5f14b1fe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:05.456097 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1a578b8f-486a-4dd4-a262-7a2e5f14b1fe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:05.456576 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-1a578b8f-486a-4dd4-a262-7a2e5f14b1fe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:05.467626 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [req-da47770a-6ad9-4e74-b817-e364a5bab0b7 req-529b92e6-182b-4e45-a297-5053a76d68cb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1812c1d4-2552-4a5e-9e55-42c7eec001cd", "tag": "f4ba7cc3-9768-4fde-9952-2e905e5b30e5"}]} {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:08:05.476958 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-da47770a-6ad9-4e74-b817-e364a5bab0b7 req-529b92e6-182b-4e45-a297-5053a76d68cb service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:05.477650 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-da47770a-6ad9-4e74-b817-e364a5bab0b7 req-529b92e6-182b-4e45-a297-5053a76d68cb service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:05.478075 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [req-da47770a-6ad9-4e74-b817-e364a5bab0b7 req-529b92e6-182b-4e45-a297-5053a76d68cb service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:05.490863 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.compute.server_external_events [req-da47770a-6ad9-4e74-b817-e364a5bab0b7 req-529b92e6-182b-4e45-a297-5053a76d68cb service nova] Creating event network-changed:f4ba7cc3-9768-4fde-9952-2e905e5b30e5 for instance 1812c1d4-2552-4a5e-9e55-42c7eec001cd on hv-cinder-maste Oct 24 09:08:05.496708 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [req-da47770a-6ad9-4e74-b817-e364a5bab0b7 req-529b92e6-182b-4e45-a297-5053a76d68cb service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.182636 Oct 24 09:08:05.497715 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 604/1217] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:08:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 24 09:08:05.597946 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.policy [None req-1a578b8f-486a-4dd4-a262-7a2e5f14b1fe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130395) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:08:05.606299 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-1a578b8f-486a-4dd4-a262-7a2e5f14b1fe tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1577 microversion: 2.1 time: 0.164560 Oct 24 09:08:05.606881 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 614/1218] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:08:05 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1577 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 09:08:06.632109 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ef2bf956-1c07-44f3-877c-572d440ac39e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:08:06.632872 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-ef2bf956-1c07-44f3-877c-572d440ac39e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:08:06.640057 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ef2bf956-1c07-44f3-877c-572d440ac39e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:06.640572 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ef2bf956-1c07-44f3-877c-572d440ac39e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:06.640998 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ef2bf956-1c07-44f3-877c-572d440ac39e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:06.641686 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ef2bf956-1c07-44f3-877c-572d440ac39e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:06.642131 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ef2bf956-1c07-44f3-877c-572d440ac39e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:06.642587 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-ef2bf956-1c07-44f3-877c-572d440ac39e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:07.115937 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-ef2bf956-1c07-44f3-877c-572d440ac39e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:08:07.124451 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ef2bf956-1c07-44f3-877c-572d440ac39e tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1577 microversion: 2.1 time: 0.497192 Oct 24 09:08:07.125080 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 605/1219] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:08:06 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1577 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 24 09:08:08.117742 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [req-9ab6182f-2fbc-432d-b9f5-5104f7fc53fe req-652a3438-a727-47c7-a450-3201cd6732d5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1812c1d4-2552-4a5e-9e55-42c7eec001cd", "name": "network-vif-deleted", "tag": "f4ba7cc3-9768-4fde-9952-2e905e5b30e5"}]} {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 24 09:08:08.128183 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-9ab6182f-2fbc-432d-b9f5-5104f7fc53fe req-652a3438-a727-47c7-a450-3201cd6732d5 service nova] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:08.128665 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-9ab6182f-2fbc-432d-b9f5-5104f7fc53fe req-652a3438-a727-47c7-a450-3201cd6732d5 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:08.129096 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [req-9ab6182f-2fbc-432d-b9f5-5104f7fc53fe req-652a3438-a727-47c7-a450-3201cd6732d5 service nova] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:08.140266 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.compute.server_external_events [req-9ab6182f-2fbc-432d-b9f5-5104f7fc53fe req-652a3438-a727-47c7-a450-3201cd6732d5 service nova] Creating event network-vif-deleted:f4ba7cc3-9768-4fde-9952-2e905e5b30e5 for instance 1812c1d4-2552-4a5e-9e55-42c7eec001cd on hv-cinder-maste Oct 24 09:08:08.145231 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [req-9ab6182f-2fbc-432d-b9f5-5104f7fc53fe req-652a3438-a727-47c7-a450-3201cd6732d5 service nova] 10.222.0.39 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.094849 Oct 24 09:08:08.145827 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 615/1220] 10.222.0.39 () {64 vars in 1317 bytes} [Mon Oct 24 09:08:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 24 09:08:08.148702 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-c3c3a161-3d5a-4706-ac32-fdf33214e2f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:08:08.149258 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-c3c3a161-3d5a-4706-ac32-fdf33214e2f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:08:08.155916 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c3c3a161-3d5a-4706-ac32-fdf33214e2f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:08.156476 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c3c3a161-3d5a-4706-ac32-fdf33214e2f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:08.156959 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c3c3a161-3d5a-4706-ac32-fdf33214e2f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:08.157664 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c3c3a161-3d5a-4706-ac32-fdf33214e2f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:08.158120 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c3c3a161-3d5a-4706-ac32-fdf33214e2f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:08.158545 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-c3c3a161-3d5a-4706-ac32-fdf33214e2f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:08.225357 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.policy [None req-c3c3a161-3d5a-4706-ac32-fdf33214e2f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3cddd27c37514e74a38430c46cef16fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30742e7b914243df8dd120079560a3de', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130396) authorize /opt/stack/nova/nova/policy.py:199}} Oct 24 09:08:08.234000 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-c3c3a161-3d5a-4706-ac32-fdf33214e2f8 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 200 len: 1224 microversion: 2.1 time: 0.089478 Oct 24 09:08:08.234554 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 606/1221] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:08:08 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 1224 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 24 09:08:09.259224 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-6b622ed4-dc46-496a-81b4-3454bfe4b9a7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:08:09.259851 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-6b622ed4-dc46-496a-81b4-3454bfe4b9a7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: 1812c1d4-2552-4a5e-9e55-42c7eec001cd] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:08:09.266035 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6b622ed4-dc46-496a-81b4-3454bfe4b9a7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:09.266513 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6b622ed4-dc46-496a-81b4-3454bfe4b9a7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:09.266951 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6b622ed4-dc46-496a-81b4-3454bfe4b9a7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:09.267640 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6b622ed4-dc46-496a-81b4-3454bfe4b9a7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:09.268099 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6b622ed4-dc46-496a-81b4-3454bfe4b9a7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:09.268618 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-6b622ed4-dc46-496a-81b4-3454bfe4b9a7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:09.282074 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-6b622ed4-dc46-496a-81b4-3454bfe4b9a7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] HTTP exception thrown: Instance 1812c1d4-2552-4a5e-9e55-42c7eec001cd could not be found. Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-6b622ed4-dc46-496a-81b4-3454bfe4b9a7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Request method failure captured: Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd HTTP/1.1 Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlQLoEuIeuhEpW8bsRGnfBcSL-MruEuqIHx01hIOb-lCoWsb18wEa-RL0_ZfZgzNX7tLN6J7zRPamBYUN5Q2h22RCUYnOaAbGpXYsIe862ctqswbh9eXLbV__Gfvp2TY03EtBVsWLXdYyHLJmVeYSUmfNZChusOQVadnrf3HbMtegdk Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: 30742e7b914243df8dd120079560a3de Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-TestVolumeBootPattern-2039901937 Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:08:09.283669 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:08:09.285379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/30742e7b914243df8dd120079560a3de"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_30742e7b914243df8dd120079560a3de", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/30742e7b914243df8dd120079560a3de"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/30742e7b914243df8dd120079560a3de"}]}] Oct 24 09:08:09.285379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-TestVolumeBootPattern-2039901937 Oct 24 09:08:09.285379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: 30742e7b914243df8dd120079560a3de Oct 24 09:08:09.285379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-TestVolumeBootPattern-2039901937 Oct 24 09:08:09.285379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-TestVolumeBootPattern-2039901937-project Oct 24 09:08:09.285379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:08:09.285379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:08:09.285379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: 3cddd27c37514e74a38430c46cef16fa Oct 24 09:08:09.285379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-TestVolumeBootPattern-2039901937-project Oct 24 09:08:09.285379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 09:08:09.285379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance 1812c1d4-2552-4a5e-9e55-42c7eec001cd could not be found. Oct 24 09:08:09.285379 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': '1812c1d4-2552-4a5e-9e55-42c7eec001cd'} Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1812c1d4-2552-4a5e-9e55-42c7eec001cd could not be found. Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:08:09.286487 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1812c1d4-2552-4a5e-9e55-42c7eec001cd could not be found. Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1812c1d4-2552-4a5e-9e55-42c7eec001cd could not be found. Oct 24 09:08:09.288109 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:08:09.289751 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-6b622ed4-dc46-496a-81b4-3454bfe4b9a7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Returning 404 to user: Instance 1812c1d4-2552-4a5e-9e55-42c7eec001cd could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:08:09.289751 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-6b622ed4-dc46-496a-81b4-3454bfe4b9a7 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd" status: 404 len: 111 microversion: 2.1 time: 0.030765 Oct 24 09:08:09.289751 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 616/1222] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:08:09 2022] GET /compute/v2.1/servers/1812c1d4-2552-4a5e-9e55-42c7eec001cd => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:08:09.720581 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f2bda435-0224-45bc-8bf6-137351f1139c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:08:09.721131 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.compute.api [None req-f2bda435-0224-45bc-8bf6-137351f1139c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130396) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:08:09.727502 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f2bda435-0224-45bc-8bf6-137351f1139c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:09.727983 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f2bda435-0224-45bc-8bf6-137351f1139c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:09.728446 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f2bda435-0224-45bc-8bf6-137351f1139c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:09.729136 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f2bda435-0224-45bc-8bf6-137351f1139c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:09.729583 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f2bda435-0224-45bc-8bf6-137351f1139c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:09.730003 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG oslo_concurrency.lockutils [None req-f2bda435-0224-45bc-8bf6-137351f1139c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:09.742893 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.wsgi [None req-f2bda435-0224-45bc-8bf6-137351f1139c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] HTTP exception thrown: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f2bda435-0224-45bc-8bf6-137351f1139c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Request method failure captured: Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: request: DELETE /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 HTTP/1.1 Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept: application/json Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Accept-Encoding: identity Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Connection: close Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Length: 0 Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Content-Type: application/json Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Host: 10.222.0.39 Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: Openstack-System-Scope: None Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: User-Agent: python-urllib3/1.26.12 Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Auth-Token: gAAAAABjVlQLoEuIeuhEpW8bsRGnfBcSL-MruEuqIHx01hIOb-lCoWsb18wEa-RL0_ZfZgzNX7tLN6J7zRPamBYUN5Q2h22RCUYnOaAbGpXYsIe862ctqswbh9eXLbV__Gfvp2TY03EtBVsWLXdYyHLJmVeYSUmfNZChusOQVadnrf3HbMtegdk Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Id: None Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Domain-Name: None Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Identity-Status: Confirmed Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Is-Admin-Project: True Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Id: default Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Domain-Name: Default Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Id: 30742e7b914243df8dd120079560a3de Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Project-Name: tempest-TestVolumeBootPattern-2039901937 Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Role: member,reader Oct 24 09:08:09.744665 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Roles: member,reader Oct 24 09:08:09.746642 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/30742e7b914243df8dd120079560a3de"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_30742e7b914243df8dd120079560a3de", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/30742e7b914243df8dd120079560a3de"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/30742e7b914243df8dd120079560a3de"}]}] Oct 24 09:08:09.746642 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant: tempest-TestVolumeBootPattern-2039901937 Oct 24 09:08:09.746642 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Id: 30742e7b914243df8dd120079560a3de Oct 24 09:08:09.746642 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-Tenant-Name: tempest-TestVolumeBootPattern-2039901937 Oct 24 09:08:09.746642 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User: tempest-TestVolumeBootPattern-2039901937-project Oct 24 09:08:09.746642 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Id: default Oct 24 09:08:09.746642 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Domain-Name: Default Oct 24 09:08:09.746642 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Id: 3cddd27c37514e74a38430c46cef16fa Oct 24 09:08:09.746642 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: X-User-Name: tempest-TestVolumeBootPattern-2039901937-project Oct 24 09:08:09.746642 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: method: > Oct 24 09:08:09.746642 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: exception: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. Oct 24 09:08:09.746642 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: action_args: {'id': 'f2c4ad60-dd40-42b5-8728-8a414e8d7644'} Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:08:09.747609 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. Oct 24 09:08:09.749013 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: ERROR nova.api.openstack.wsgi Oct 24 09:08:09.750906 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-f2bda435-0224-45bc-8bf6-137351f1139c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Returning 404 to user: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. {{(pid=130396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:08:09.750906 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-f2bda435-0224-45bc-8bf6-137351f1139c tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "DELETE /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 404 len: 111 microversion: 2.1 time: 0.029342 Oct 24 09:08:09.750906 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 607/1223] 10.222.0.39 () {58 vars in 1239 bytes} [Mon Oct 24 09:08:09 2022] DELETE /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 24 09:08:09.755009 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ac479bfb-d103-4475-b6cb-0b25cb83cbc3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '>' {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:08:09.755621 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.compute.api [None req-ac479bfb-d103-4475-b6cb-0b25cb83cbc3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] [instance: f2c4ad60-dd40-42b5-8728-8a414e8d7644] Fetching instance by UUID {{(pid=130395) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 24 09:08:09.762096 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ac479bfb-d103-4475-b6cb-0b25cb83cbc3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:09.762582 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ac479bfb-d103-4475-b6cb-0b25cb83cbc3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:09.763013 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ac479bfb-d103-4475-b6cb-0b25cb83cbc3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:09.763703 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ac479bfb-d103-4475-b6cb-0b25cb83cbc3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Acquiring lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 24 09:08:09.764196 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ac479bfb-d103-4475-b6cb-0b25cb83cbc3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 24 09:08:09.764621 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG oslo_concurrency.lockutils [None req-ac479bfb-d103-4475-b6cb-0b25cb83cbc3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Lock "0c69bc68-c2d9-408a-9288-c4e45fffaab5" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130395) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 24 09:08:09.778182 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.wsgi [None req-ac479bfb-d103-4475-b6cb-0b25cb83cbc3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] HTTP exception thrown: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ac479bfb-d103-4475-b6cb-0b25cb83cbc3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Request method failure captured: Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: request: GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 HTTP/1.1 Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept: application/json Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Accept-Encoding: identity Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Connection: close Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Length: 0 Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Content-Type: application/json Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Host: 10.222.0.39 Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: Openstack-System-Scope: None Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: User-Agent: python-urllib3/1.26.12 Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Auth-Token: gAAAAABjVlQLoEuIeuhEpW8bsRGnfBcSL-MruEuqIHx01hIOb-lCoWsb18wEa-RL0_ZfZgzNX7tLN6J7zRPamBYUN5Q2h22RCUYnOaAbGpXYsIe862ctqswbh9eXLbV__Gfvp2TY03EtBVsWLXdYyHLJmVeYSUmfNZChusOQVadnrf3HbMtegdk Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Id: None Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Domain-Name: None Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Identity-Status: Confirmed Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Is-Admin-Project: True Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Id: default Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Domain-Name: Default Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Id: 30742e7b914243df8dd120079560a3de Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Project-Name: tempest-TestVolumeBootPattern-2039901937 Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Role: member,reader Oct 24 09:08:09.779671 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Roles: member,reader Oct 24 09:08:09.781473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2/30742e7b914243df8dd120079560a3de"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:9696/networking"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39:8080/v1/AUTH_30742e7b914243df8dd120079560a3de", "adminURL": "http://10.222.0.39:8080"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/30742e7b914243df8dd120079560a3de"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.39/volume/v3/30742e7b914243df8dd120079560a3de"}]}] Oct 24 09:08:09.781473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant: tempest-TestVolumeBootPattern-2039901937 Oct 24 09:08:09.781473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Id: 30742e7b914243df8dd120079560a3de Oct 24 09:08:09.781473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-Tenant-Name: tempest-TestVolumeBootPattern-2039901937 Oct 24 09:08:09.781473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User: tempest-TestVolumeBootPattern-2039901937-project Oct 24 09:08:09.781473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Id: default Oct 24 09:08:09.781473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Domain-Name: Default Oct 24 09:08:09.781473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Id: 3cddd27c37514e74a38430c46cef16fa Oct 24 09:08:09.781473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: X-User-Name: tempest-TestVolumeBootPattern-2039901937-project Oct 24 09:08:09.781473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: method: > Oct 24 09:08:09.781473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: exception: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. Oct 24 09:08:09.781473 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: action_args: {'id': 'f2c4ad60-dd40-42b5-8728-8a414e8d7644'} Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: {{(pid=130395) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 24 09:08:09.782541 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. Oct 24 09:08:09.783894 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: ERROR nova.api.openstack.wsgi Oct 24 09:08:09.785532 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: DEBUG nova.api.openstack.wsgi [None req-ac479bfb-d103-4475-b6cb-0b25cb83cbc3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Returning 404 to user: Instance f2c4ad60-dd40-42b5-8728-8a414e8d7644 could not be found. {{(pid=130395) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 24 09:08:09.785532 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: INFO nova.api.openstack.requestlog [None req-ac479bfb-d103-4475-b6cb-0b25cb83cbc3 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644" status: 404 len: 111 microversion: 2.1 time: 0.030243 Oct 24 09:08:09.785532 dv-cinder-master-3-iscsi devstack@n-api.service[130395]: [pid: 130395|app: 0|req: 617/1224] 10.222.0.39 () {58 vars in 1236 bytes} [Mon Oct 24 09:08:09 2022] GET /compute/v2.1/servers/f2c4ad60-dd40-42b5-8728-8a414e8d7644 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 24 09:08:11.075746 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: DEBUG nova.api.openstack.wsgi [None req-ec464800-8df1-4b8e-a268-16170c122de6 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] Calling method '.version_select at 0x7fdfedcc0310>' {{(pid=130396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 24 09:08:11.097086 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: INFO nova.api.openstack.requestlog [None req-ec464800-8df1-4b8e-a268-16170c122de6 tempest-TestVolumeBootPattern-2039901937 tempest-TestVolumeBootPattern-2039901937-project] 10.222.0.39 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-593019647" status: 202 len: 0 microversion: 2.1 time: 0.024461 Oct 24 09:08:11.098074 dv-cinder-master-3-iscsi devstack@n-api.service[130396]: [pid: 130396|app: 0|req: 608/1225] 10.222.0.39 () {58 vars in 1260 bytes} [Mon Oct 24 09:08:11 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-593019647 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0)